Building inspection based on polarized images

Abstract

A system of the present disclosure enables building inspections using polarized images. The system detects misapplication of tape applied to a substrate using light polarization information of the polarized images. An exemplary system includes polarization camera hardware, a memory communicatively coupled to the polarization camera hardware, and a processing circuit communicatively coupled to the memory. The polarization camera hardware is configured to capture a polarized image of the tape applied to the substrate. The memory is configured to store the polarized image. The processing circuit is configured to analyze the polarized image according to a trained classification model and detect misapplication of the tape applied to the substrate based on the analysis of the polarized image according to the trained classification model.

Description

Technical Field

[0001] The present disclosure generally relates to the field of construction-related functions implemented using drones.

Background Art

[0002] During or after construction, buildings are inspected. If certain defects are detected during such inspections, the defects can be marked and / or repaired for future identification, whether at the construction stage or after completion.

Summary of the Invention

[0003] The present disclosure describes a system configured for building inspection, building defect marking, and building defect repair using drones. The present disclosure is mainly described with the drone-hosted technology being performed on the building outer skin layer during building construction as a non-limiting example. However, it will be understood that the various drone-hosted technologies of the present disclosure are applicable to various aspects of the building, whether the building is currently under construction or fully constructed. Some embodiments of the present disclosure utilize camera hardware incorporated in the drone to acquire one or more images of the building (e.g., of the building outer skin). According to these embodiments, the system of the present disclosure analyzes the image(s) using a trained machine-learning (ML) model and determines whether a portion of the building shown in the image(s) contains a defect that the ML model is trained to detect.

[0004] Some embodiments of the present disclosure relate to drone-hosted marking operations for building defects. In these embodiments, the drone may include or be coupled to a marking subsystem, such as an ink supply subsystem or a self-tack paper supply subsystem. In these embodiments, the system of the present disclosure can activate the marking subsystem to mark an area of ​​the identified defect or an area near it. Some embodiments of the present disclosure relate to drone-hosted repair operations for building defects. In these embodiments, the drone may include or be coupled to a repair subsystem, such as an aerosol supply subsystem or an adhesive supply subsystem. In these embodiments, the system of the present disclosure can activate the repair subsystem to supply aerosol or adhesive (optionally) to an area associated with the identified defect.

[0005] In one embodiment, the system includes polarizing camera hardware, a memory communicatively coupled to the polarizing camera hardware, and a processing circuit communicatively coupled to the memory. The polarizing camera hardware is configured to capture polarized images of tape applied to a substrate. The memory is configured to store the polarized images. The processing circuit is configured to analyze the polarized images according to a trained classification model and to detect misapplication of the tape applied to the substrate based on the analysis of the polarized images according to the trained classification model.

[0006] In another embodiment, the method includes capturing a polarized image of a tape applied to a substrate using polarized camera hardware. The method further includes analyzing the polarized image according to a trained classification model by a processing circuit communicatively coupled to the image capture hardware. The method further includes detecting misapplication of the tape applied to the substrate based on the analysis of the polarized image according to the trained classification model by the processing circuit.

[0007] In another embodiment, the apparatus includes means for capturing a polarized image of a tape applied to a substrate, means for analyzing the polarized image according to a trained model, and means for detecting misapplication of the tape applied to the substrate based on the analysis of the polarized image according to the trained model.

[0008] In another embodiment, a computer-readable memory device has instructions encoded within it. When these instructions are executed, the processing circuit of the computing device receives a polarized image of the tape applied to the substrate from the polarized camera hardware, stores the polarized image in the computer-readable memory device, analyzes the polarized image according to a trained model, and detects misapplication of the tape applied to the substrate based on the analysis of the polarized image according to the trained model.

[0009] The system of this disclosure offers several potential advantages over currently available solutions. By hosting image capture, defect marking, and defect repair operations on a drone, the system of this disclosure improves data accuracy by improving safety and reducing the occurrence of human error when workers are deployed to sites where weather / visibility conditions may fluctuate and which may be at high altitudes. The defect detection technology of this disclosure analyzes image data of building areas by running a trained ML model (which may be a classification model, detection model, or segmentation model in the various embodiments of this disclosure) to reduce the likelihood of human error where safety concerns are critical.

[0010] Furthermore, the drone-hosted technology of this disclosure can improve the accuracy and completeness of inspections, markings, or repairs by leveraging the maneuverability of a drone to more thoroughly inspect buildings (or other structures) and perform inspections, markings, or repairs in areas that may be difficult for human workers to reach. In some embodiments, the drone of this disclosure is equipped with dedicated image capture hardware that provides images that can be analyzed by the trained model of this disclosure with greater accuracy than the human eye can interpret standard images or direct views of a building. In this way, the drone-hosted technology of this disclosure can improve data accuracy and / or process completeness, while also providing a practical application of enhanced safety.

[0011] Details of one or more embodiments of this disclosure are described in the accompanying drawings and the following specification. Other features, purposes, and advantages of this disclosure will become apparent from the description and drawings and the claims. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram illustrating one embodiment of the system, the embodiment of which is configured to perform one or more of the technologies of the present disclosure.

[0013] [Figure 2] This is a conceptual diagram illustrating an embodiment of the drone-host type tape application inspection method disclosed herein.

[0014] [Figure 3A] Figure 1 is a conceptual diagram illustrating further details of the system configuration that can be detected using the technology of this disclosure, specifically regarding the misapplication of tape to a substrate. [Figure 3B] Figure 1 is a conceptual diagram illustrating further details of the system configuration that can be detected using the technology of this disclosure, specifically regarding the misapplication of tape to a substrate.

[0015] [Figure 4A]A diagram showing various deep learning generated image labels that can be generated by the trained classification model of the present disclosure. [Figure 4B] A diagram showing various deep learning generated image labels that can be generated by the trained classification model of the present disclosure. [Figure 4C] A diagram showing various deep learning generated image labels that can be generated by the trained classification model of the present disclosure. [Figure 4D] A diagram showing various deep learning generated image labels that can be generated by the trained classification model of the present disclosure.

[0016] [Figure 5] A conceptual diagram showing a polarization image that can be analyzed by the system of FIG. 1 to detect defects related to a tape applied to a substrate according to an aspect of the present disclosure.

[0017] [Figure 6A] A diagram showing various deep learning generated image labels that can be generated by the trained classification model of the present disclosure using the polarization image shown in FIG. 5. [Figure 6B] A diagram showing various deep learning generated image labels that can be generated by the trained classification model of the present disclosure using the polarization image shown in FIG. 5. [Figure 6C] A diagram showing various deep learning generated image labels that can be generated by the trained classification model of the present disclosure using the polarization image shown in FIG. 5. [Figure 6D] A diagram showing various deep learning generated image labels that can be generated by the trained classification model of the present disclosure using the polarization image shown in FIG. 5.

[0018] [Figure 7] A graph showing an aspect of polarization image analysis that can be performed by the trained classification model of the present disclosure to detect one or more defects related to a tape applied to a substrate.

[0019] [Figure 8]It is a conceptual diagram showing an aspect of drone-hosted substrate inspection of the present disclosure.

[0020] [Figure 9A] It is a conceptual diagram showing an example of an underdrive fastener in a substrate that can be detected as a substrate defect by the trained classification model of the present disclosure. [Figure 9B] It is a conceptual diagram showing an example of an underdrive fastener in a substrate that can be detected as a substrate defect by the trained classification model of the present disclosure. [Figure 9C] It is a conceptual diagram showing an example of an underdrive fastener in a substrate that can be detected as a substrate defect by the trained classification model of the present disclosure.

[0021] [Figure 10A] It is a conceptual diagram showing an example of board separation in a substrate that can be detected as a substrate defect by the trained classification model of the present disclosure. [Figure 10B] It is a conceptual diagram showing an example of board separation in a substrate that can be detected as a substrate defect by the trained classification model of the present disclosure.

[0022] [Figure 11] It is a conceptual diagram showing an example of a substrate defect caused by an overdrive fastener and detected using the trained classification model of the present disclosure.

[0023] [Figure 12A] It is a conceptual diagram showing an example of collision-related damage in a substrate that can be detected as a substrate defect by the trained classification model of the present disclosure. [Figure 12B] It is a conceptual diagram showing an example of collision-related damage in a substrate that can be detected as a substrate defect by the trained classification model of the present disclosure.

[0024] [Figure 13A]This figure shows an embodiment according to the present disclosure in which a drone is equipped and configured to mark potential objects of investigation on a substrate or tape. [Figure 13B] This figure shows an embodiment according to the present disclosure in which a drone is equipped and configured to mark potential objects of investigation on a substrate or tape.

[0025] [Figure 14A] This is a conceptual diagram showing an embodiment according to the present disclosure in which a drone is equipped and configured to repair a potential object of investigation on a substrate or tape applied to a substrate. [Figure 14B] This is a conceptual diagram showing an embodiment according to the present disclosure in which a drone is equipped and configured to repair a potential object of investigation on a substrate or tape applied to a substrate. [Figure 14C] This is a conceptual diagram showing an embodiment according to the present disclosure in which a drone is equipped and configured to repair a potential object of investigation on a substrate or tape applied to a substrate.

[0026] [Figure 15] This is a conceptual diagram illustrating another embodiment according to aspects of the present disclosure, in which a drone is equipped and configured to repair a potential object of investigation on a substrate or tape applied to a substrate.

[0027] [Figure 16] This is a flowchart illustrating an exemplary process in this disclosure.

[0028] It should be understood that embodiments can be utilized and structural modifications can be made without departing from the scope of the present invention. These figures are not necessarily to a constant scale. Similar numbers used in the drawings indicate similar components. However, it should be understood that the use of numbers to indicate components in a given figure is not intended to limit components in other figures indicated by the same numbers. [Modes for carrying out the invention]

[0029] Figure 1 is a conceptual diagram showing one embodiment of System 10, which is configured to perform one or more of the technologies of this disclosure. System 10 includes a building 2, a drone 4, a drone controller 6, and a computing system 8. Building 2 is shown as being in the construction phase, during which the exposed outer layer is the “skin layer” or “building envelope.” While the technologies of this disclosure are described as being performed on the building envelope in non-limiting embodiments, it will be understood that various technologies of this disclosure are equally applicable to other substrates. Examples of other substrates include non-building structures such as completed building walls, walls, fences, bridges, ships, aircraft, and cell phone towers, whether outdoors or indoors.

[0030] The building envelope refers to the physical barrier between the regulated and unregulated environments of each building (in this case, Building 2). In various embodiments, the building envelope may be called the "building enclosure," the "skin layer" as described above, or the "weatherproof barrier" (WPB). The building envelope shields the interior of the building from outdoor elements and plays a crucial role in environmental control. The elements shielding and environmental control functions of the building envelope include rain shielding, air control, heat transfer control, and vapor shielding. Therefore, the integrity of the building envelope is essential for the safety and habitability of Building 2.

[0031] With regard to tasks such as inspecting building envelope defects, marking any defects found, or repairing any defects found in the building envelope, accuracy and completeness are critical to the goal of maintaining integrity. As building size, design complexity, and density increase, performing these tasks manually is becoming increasingly difficult. System 10 can leverage the maneuverability of a drone (e.g., Drone 4) to perform one or more of the following: building envelope inspection, defect marking, and / or defect repair. System 10 can also leverage dedicated computing power to identify the presence of potential defects, their location if any, and / or the parameters of the actions to be taken to repair any such potential defects. This dedicated computing power can be provided by one or more computing or processing hardware components from Drone 4, Drone Controller 6, and / or Computing System 8. In some embodiments, aspects of System 10 can leverage cloud computing resources to implement dedicated computing power in a distributed manner.

[0032] Drone 4 can represent one or more types of unmanned aerial vehicles (UAVs). In various embodiments, drone 4 may also be called one or more of the following: autonomous aircraft, autopilot vehicle, remotely operated aircraft, remotely piloted aircraft, remotely piloted aircraft system, remotely piloted aircraft system, remotely piloted aircraft system, remotely piloted aircraft vehicle, remotely piloted system, remotely piloted vehicle, small unmanned aircraft system, small unmanned aircraft, unmanned flight system, unmanned flight vehicle, remotely piloted transport system, etc.

[0033] The processing circuit of the drone controller 6 and / or the processing circuit of the computing system 8 can formulate navigation commands for the drone 4 based on the location of the area of ​​building 2 that is subject to inspection, defect marking, or defect repair by the drone 4 and its respective subsystems. The processing circuit can then optionally activate the wireless interface hardware of the drone controller 6 or the computing system 8 to transmit navigation commands to the wireless interface hardware of the drone 4. The wireless interface hardware of the drone 4, the drone controller 6, and the computing system 8 can represent communication hardware that enables wireless communication between two or more of the drone 4, the drone controller 6, and / or the computing system 8, and also enables wireless communication with other devices that have wireless interface hardware.

[0034] The drone 4 may be equipped with motion guides to control the movement of the drone 4, such as the flight path of the drone 4. The drone 4 may also be equipped with control logic to receive navigation commands from either the drone controller 6 or the computing system 8 via the drone 4's wireless interface hardware. The control logic can use the navigation commands received from the drone controller 6 or the computing system 8 to navigate the drone 4 to an area adjacent to a specific part of the building 2. In various embodiments consistent with the aspects of this disclosure, the processing circuits of the drone controller 6 and / or the computing system 8 may form navigation commands based on the area of ​​the building 2 being inspected for objects of potential survey interest (OoPSI), or based on an area associated with a previously identified OoPSI, in order to facilitate the marking and / or restoration of identified objects of potential survey interest (OoPSI).

[0035] Computing system 8 may include, or be part of, one or more of various types of computing devices, including, in particular, mobile phones (e.g., smartphones), tablet computers, netbooks, laptop computers, desktop computers, personal digital assistants ("PDAs"), and wearable devices (e.g., smartwatches or smart glasses). In some embodiments, computing system 8 may represent a distributed system including an interconnected network of two or more such devices. In a non-limiting embodiment according to aspects of the present disclosure, computing system 8 is shown in Figure 1 as a laptop computer.

[0036] In many embodiments, the drone controller 6 represents a wireless control transmitter or transceiver. The drone controller 6 is configured to process user input received via various input hardware (e.g., joysticks, buttons, etc.), formulate the navigation commands described above, and transmit the navigation commands to the drone 4's communication interface hardware (e.g., receiver) in substantially real time via the communication interface hardware. The complementary communication interfaces of the drone 4 and the drone controller 6 can communicate via one or more predetermined frequencies.

[0037] In this way, the configuration of system 10 utilizes the flight capabilities and maneuverability of the drone 4 to inspect building 2, and in some scenarios,

number

[0038] Figure 2 is a conceptual diagram illustrating an embodiment of the drone-hosted tape application inspection of the present disclosure. Figure 2 shows a substrate 16 which in some embodiments can represent a portion of the exterior surface of a building 2, such as a wall or roof (as shown). In this embodiment, the substrate 16 comprises a tape 14. The tape 14 can represent any of various types of adhesive coating materials. The present disclosure primarily describes non-limiting embodiments in which the tape 14 represents a so-called “flushing tape” commonly used to seal seams, cracks, or other discontinuities on the exterior surface of a building, such as the substrate 16. In some embodiments, the substrate 16 can represent a surface coated with an adhesive layer, such as a roll-on adhesive, which leaves an outward-facing adhesive layer on the surface. Features of one or more non-limiting embodiments of tape 14, a sealant on which substrate 16 can be coated, and / or an outward-facing adhesive on which substrate 16 can be coated are described in published U.S. patent applications with publication numbers US20130139953A1, US2020003098A1, and US20190031923A1, as well as publication numbers WO2021033111A1, W02021024206A1, and WO201601 These are described in international patent applications having names 9248A1, WO2016106273A1, WO2015183354A2, WO2015126931A1, WO2017031275A1, W02019152621A1, WO2017112756A1, WO2017031275A1, WO2018156631A1, and WO2018220555A1, the entirety of each of these disclosures is incorporated herein by reference.

[0039] In the embodiment shown in Figure 2, the drone 4 includes image capture hardware 12. In some embodiments, the image capture hardware 12 represents one or more types of digital cameras, such as cameras configured to store captured still images and / or videos in a digital format (e.g., as .jpeg, .png, .mp4 files, etc.). Based on navigation commands received from the drone controller 6 or computing system 8, the control logic of the drone 4 can cause the motion guide to navigate the drone 4 to an area adjacent to a specific area of ​​the tape 14 applied to the substrate 16. When the control logic detects that the drone 4 is positioned and directed sufficiently to allow image capture by the image capture hardware 12, it can activate the image capture hardware 12 to capture one or more images of portions of the tape 14 that are within the field of view of the lens hardware of the image capture hardware 12.

[0040] According to some implementations consistent with this disclosure, if the image capture hardware 12 is a separate camera physically coupled to the drone 4, the control logic can operate an actuator subassembly of the drone 4 to activate or press a button on the image capture hardware 12. According to other implementations consistent with this disclosure, if the image capture hardware 12 is integrated into the drone 4, the control logic can operate the logic of the image capture hardware 12 to activate its image capture capability. The image capture hardware 12 can then provide the captured digital images(s) to the processing circuits of the drone 4 and / or the processing circuits of the computing system 8 via various types of communication channels suitable for transferring digital image data using wireless or wired means.

[0041] When used herein, the processing circuit may include one or more of the following: a central processing unit (CPU), a graphics processing unit (GPU), a single-core or multi-core processor, a controller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a fixed-function circuit, a programmable circuit, any combination of fixed-function and programmable circuits, a discrete logic circuit, or an integrated logic circuit. The processing circuit of the drone 4 or computing system 8 can analyze one or more images received from the image capture hardware 12 according to a trained ML model and, based on the analysis, can detect misapplication of the tape 14 (or a portion thereof) applied to the substrate 16. In various embodiments of this disclosure, the processing circuit may run a trained classification model, a trained detection model, or a trained segmentation model.

[0042] In some embodiments, the processing circuit of the drone 4 or computing system 8 can leverage cloud computing capabilities to run a trained model. In various embodiments, the trained model may be a trained deep learning model, such as a deep neural network. One embodiment of a trained deep neural network that the processing circuit can run to analyze images of tape 14 in accordance with this disclosure is a trained convolutional neural network (CNN) that applies computer vision-oriented machine learning techniques to detect misapplication of tape 14 applied to substrate 16. Embodiments of the CNN are described in international patent applications with publication numbers WO2021 / 033061A1 and WO2020 / 003150A2, the entirety of each of these disclosures is incorporated herein by reference. One embodiment of a trained CNN that the processing circuit of the drone 4 or computing system 8 can run to perform the defect detection embodiments of this disclosure is a Mask R-CNN.

[0043] Figures 3A and 3B are conceptual diagrams illustrating further details of misapplications of tape 14 to a substrate 16 that can be detected by an embodiment of system 10 using the technology of the present disclosure. Figure 3A shows an example of a tape application defect that can be detected by a processing circuit of a drone 4 or computing system 8 using a trained model of the present disclosure. Some of the defects shown in Figure 3A correspond to what is referred to herein as a “fishmouth wrinkle.” As used herein, a fishmouth wrinkle refers to a misapplication of tape with an unglued portion of the edge of the tape 14 from the substrate 16, having adjacent unglued or non-coplanar portions of the inner portion of the tape 14 with respect to the substrate 16. The openings created by the unglued portion of the edge of the tape 14 applied to the substrate 16 and adjacent protruding wrinkles form openings into which fluids (such as air or water) can enter and impair the function of the tape 14 applied to the substrate 16.

[0044] Another type of defect that can be detected by the processing circuit of the drone 4 or computing system 8 running a trained model includes “tenting,” which refers to the non-adhesive portion and protrusion of the inner projection of the tape 14 to the substrate 16 (e.g., without the fishmouth wrinkle edge penetration point shown in Figure 3A). Tenting of the tape 14 can be caused by incorrect application procedures or by applying the tape 14 over an “underdrive fastener.” An underdrive fastener refers to a nail, screw, bolt, or other type of fastener that is partially driven into the substrate 16, but whose head protrudes from the substrate 16 to such an extent that it causes the tape 14 to tent when applied over the head of the fastener.

[0045] Figure 3B shows another example of a tape application defect that can be detected by the processing circuitry of Drone 4 or the computing system 8 using a trained model of the present disclosure. The defect shown in Figure 3A is referred to herein as a “missing tape segment”. A missing tape segment 18 represents a misapplication of tape 14 in which a portion of the substrate 16 is left exposed when it should not be exposed, such as due to indoor adjustment issues. For example, a missing tape segment 18 may expose a substrate seam or substrate patch that should not be exposed to the element. The processing circuitry of Drone 4 or the computing system 8 can also run a trained model of the present disclosure to detect discontinuities in tape 14 other than missing tape segments 18. In one embodiment, the processing circuitry of Drone 4 or the computing system 8 can run a trained model of the present disclosure to detect a tear that does not extend across the entire width of tape 14, or a scratch that does not expose the substrate 16 beneath tape 14, but instead impairs or reduces the effectiveness of tape 14. The processing circuit of the drone 4 or computing system 8 can also run the trained model of this disclosure to analyze images received from the image capture device 12 and detect other types of misapplication of the tape 14 applied to the substrate 16, such as insufficient adhesion and insufficient tension (for example, because insufficient force when the tape 14 is applied to the substrate 16 can cause "slack" in the tape 14). In some embodiments, the processing of the drone 4 or computing system 8 (whether local or leveraging cloud computing resources) can analyze images of the tape 14 to determine brand information, standards compliance information, etc., relating to the tape 14, whether the information is represented in a human-readable manner without image processing.

[0046] Figures 4A to 4D show various deep learning-generated image labels that the trained model of the present disclosure can generate. Each of Figures 4A to 4D shows a model output that can be generated by running the trained model of the present disclosure at various levels of computational complexity on the processing circuit of the drone 4 or computing device 8. Figures 4A to 4D are shown in ascending order of computational complexity for generating the image labels of the present disclosure.

[0047] In the embodiment of Figure 4A, the trained classification model of the Disclosure performs full image classification on the image of the tape 14 applied to the substrate 16. In this embodiment, the processing circuit of the drone 4 or computing system 8 returns a “fail” classification in the form of an image label 19. The “fail” result of the image label 19 is a result or model output generated by the trained classification model based on the detection of at least one defect anywhere in the image received from the image capture hardware 12. In the embodiment of Figure 4B, the trained detection model of the Disclosure performs sub-image classification on the image of the tape 14 applied to the substrate 16. In this embodiment, the trained detection model of the Disclosure divides the image into a plurality of sub-images 20 and classifies each sub-image 20 with a “pass” or “fail” label, as done with the image label 19 for the entire image as a whole in the embodiment of Figure 4A.

[0048] In the embodiment shown in Figure 4C, the trained detection model of the present disclosure performs object detection on an image of the tape 14 applied to a substrate 16. According to the object detection-based technique shown in Figure 4C, the trained detection model of the present disclosure returns rectangular bounding boxes (bounding boxes 22) around areas of the image indicating defects in the tape 14 applied to the substrate 16, and therefor the trained detection model of the present disclosure returns each “fail” result as a model output. As shown in the non-limiting embodiment of Figure 4C, the trained detection model can perform object detection to return multiple (potentially overlapping, as in Figure 4C) bounding boxes 22.

[0049] In the embodiment shown in Figure 4D, the trained segmentation model of the Disclosure performs image segmentation on an image of the tape 14 applied to a substrate 16. According to the image segmentation-based technique shown in Figure 4D, the trained segmentation model of the Disclosure returns pixel-by-pixel labeling of defects in the tape 14 applied to the substrate 16, as represented in an image acquired from image capture hardware 12. Figure 4D shows a defect segment 24 identified by the trained segmentation model of the Disclosure based on a pixel-by-pixel analysis of an image received from image capture hardware 12 showing the tape 14 applied to the substrate 16.

[0050] Figures 3A to 4D represent images represented in various color spaces, such as the red-green-blue (RGB) color space, grayscale color space, black and white space, or various other chromaticity spaces that are partially or entirely distinguishable to the human eye, in various embodiments. In these embodiments, the image capture device 12 represents digital camera hardware configured to generate digital image data in one of these color spaces. In other embodiments, the image capture hardware 12 may represent a so-called "polarizing camera." A polarizing camera can generate image data in various formats by performing calculations on data output by the polarizing sensor of the polarizing camera.

[0051] Figure 5 is a conceptual diagram showing a polarized image that the system 10 can analyze to detect defects in the tape 14 applied to the substrate 16, according to an embodiment of the present disclosure. In embodiments where the image capture hardware 12 represents a polarized camera, the processing circuit of the drone 4 or computing system 8 can analyze various data output by the image capture hardware 12 to detect defects in the tape 14 applied to the substrate 16. For example, the processing circuit of the drone 4 or computing system 8 can run the trained classification model of the present disclosure to analyze two values: unpolarized in the polarized image and the degree of linear polarization (DoLP) shown in the polarized image. Figure 5 shows a pre-processed or processed polarized image including DoLP data calculated for each pixel in each color channel, which can be used by the trained model of the present disclosure to detect application defects based on illustrated wrinkles in the tape 14 with high confidence.

[0052] In some non-limiting embodiments where the image capture hardware represents a polarizing camera, the image sensor hardware of the image capture hardware 12 includes four polarizing filters directed at 0 degrees, 45 degrees, 90 degrees, and 135 degrees, respectively. Using a notation in which the four images obtained through the four polarizing filters are represented by I0, I45, I90, and I135, the polarizing camera of the image capture hardware 12 can calculate Stokes vectors (S0, S1, S2, and S3) for each color channel according to the following calculation.

number

[0053] The Stokes vectors calculated according to equation (1) above represent, respectively, the unpolarized intensity image (S0), the linearly or horizontally polarized intensity image (S1), the intensity image of light polarized at 45 or 135 degrees (S2), and the circularly polarized light (S3). The polarization camera can calculate DoLP using the above Stokes vectors according to equation (2) below.

number

[0054] The processing circuit of the drone 4 or computing system 8 can run a trained model of the present disclosure and use linear polarization to detect wrinkle-based defects, such as wrinkles 26 and / or fishmouth wrinkles 28, on the tape 14 applied to the substrate 16. Since the surface of the tape 14 reflects light differently depending on the angle of the incident light being reflected, the presence and potentially size (with respect to the angle outward from the substrate 16) of the wrinkles 26 and / or fishmouth wrinkles 28 will change the polarization-based measurement described above.

[0055] When run, the trained model of this disclosure can measure the consistency of light reflection without regard to the direction of light reflection using DoLP calculated according to equation (2). In various experiments, the tape 14 had a black and glossy appearance. Regardless of the color of the tape 14, the trained model of this disclosure can leverage the glossy properties of the tape 14 to use DoLP to detect wrinkle shading and other effects and detect defects in the application of the tape 14 to the substrate 16. Darker colors of the tape 14 (such as the black used in the above experiments) can further enhance the ability of the trained model of this disclosure to detect wrinkles 26 and / or fishmouth wrinkles 28 of the tape 14 applied to the substrate 16 using DoLP.

[0056] Figures 6A to 6D show various deep learning-generated image labels that the trained model of the present disclosure can generate using the polarized images shown in Figure 5. Each of Figures 6A to 6D shows an image label that can be generated by running the trained model of the present disclosure on the polarized images of Figure 5 at various levels of computational complexity on the processing circuit of the drone 4 or computing device 8. Figures 6A to 6D are shown in ascending order of computational complexity for generating image labels using the polarized images of Figure 5.

[0057] In the embodiment shown in Figure 6A, the trained classification model of the present disclosure performs full image classification on a polarized image of the tape 14 applied to a substrate 16. In this embodiment, the processing circuit of the drone 4 or computing system 8 returns a “fail” result based on a full image analysis of the polarized image, the conditions met when the wrinkles 26 or fishmouth wrinkles 28 are first detected, or in the case of substantially simultaneous detection. The “fail” result of the model output is indicated by the image label 29 in Figure 6A.

[0058] In the embodiment shown in Figure 6B, the trained detection model of the Disclosure performs sub-image classification on a polarized image of the tape 14 applied to a substrate 16. In this embodiment, the trained detection model of the Disclosure divides the polarized image into a plurality of sub-images 30 and classifies each sub-image 30 as either a "pass" or "fail" result, as was done on the entire polarized image as a whole in the embodiment shown in Figure 6A.

[0059] In the embodiment shown in Figure 6C, the trained detection model of the present disclosure performs object detection on a polarized image of the tape 14 applied to a substrate 16. According to the object detection-based technique shown in Figure 6C, the trained detection model of the present disclosure returns a rectangular bounding box (bounding box 32) around an area of ​​the polarized image indicating a defect in the tape 14 applied to the substrate 16. As shown in the non-limiting embodiment of Figure 6C, the trained detection model can perform object detection to return multiple (and potentially overlapping) bounding boxes 32.

[0060] In the embodiment shown in Figure 6D, the trained segmentation model of the Disclosure performs image segmentation on a polarized image of the tape 14 applied to a substrate 16. According to the image segmentation-based technique shown in Figure 6D, the trained segmentation model of the Disclosure returns pixel-by-pixel labeling of wrinkles 26 and fishmouth wrinkles 28 on the tape 14 applied to the substrate 16, as represented in the image acquired from the polarized camera of the image capture hardware 12. Figure 6D shows a defective segment 24 that the trained segmentation model of the Disclosure identifies based on a pixel-by-pixel analysis of the optical polarization shown in the polarized image of Figure 5 showing the tape 14 applied to the substrate 16.

[0061] Figure 7 is a graph 36 illustrating an embodiment of polarization image analysis that the trained model of the present disclosure can perform to detect one or more defects in a tape 14 applied to a substrate 16. Plot lines 38, 40, and 42 show the change in the true positive rate (for image classification) as a function of the corresponding false positive rate under different polarization image analysis scenarios. Overall, graph 36 represents receiver operator characteristic (ROC) curves generated from a test dataset classified by the trained model of the present disclosure with respect to DoLP and unpolarized (S0) images.

[0062] As shown in Graph 36, the area under the curve (AUC) is highest for the DoLP plot line 38 (corresponding to the polarized image) compared to the AUC for the S0 plot line 40 (corresponding to the unpolarized image) and the random plot line 42 (provided as the baseline ground truth). The AUC shown in Figure 7 provides an overall measure of how distinguishable the corresponding classes of images are in the dataset provided to the trained model of this disclosure. The AUC for the DoLP plot line 38 is the highest among the plot lines shown in Graph 38, indicating that the trained model of this disclosure can distinguish wrinkles 26 and / or fishmouth wrinkles 28 from DoLP images more easily than when using the other images.

[0063] In this way, the technology of the Disclosure improves the accuracy of defect detection of the tape 14 applied to the substrate 16 by using the trained model of the Disclosure (e.g., one or more of the classification, detection, or segmentation models). Regardless of whether the image capture hardware 12 provides images in RGB color space, grayscale color space, or as polarized images (or DoLP images), the trained model of the Disclosure detects various types of defects in the tape 14 applied to the substrate 16 while improving data accuracy (e.g., by mitigating human errors resulting from different visual or perceptual abilities). Although primarily described as being coupled to or incorporated into a drone 4 as an example of a non-limiting use case, it will be understood that the image analysis technology based on the trained model of the Disclosure also provides these data accuracy improvements in non-drone-based implementations as well.

[0064] For example, the trained model of the present disclosure can use images captured by the image capture hardware 12 in an embodiment in which the image capture hardware 12 is incorporated into a mobile computing device such as a smartphone, tablet computer, or wearable computing device. In another embodiment, the trained model of the present disclosure can use images captured by the image capture device 12 if the image capture device 12 is a dedicated digital camera or a dedicated polarizing camera. In any of the non-drone-based embodiments listed above, the trained model of the present disclosure can use images of the tape 14 applied to the substrate 16 based on manual capture of images, such as by user input provided via an actuator button on a digital camera or touch input provided on a touchscreen of a mobile computing device.

[0065] According to the drone-hosted implementation described above, the system of this disclosure improves safety and also improves the ability to capture and analyze images from hard-to-reach areas of the substrate 16. For example, by using a drone 4 to transport the image capture hardware 12 to potentially hazardous locations and capturing images at these locations, the system 10 reduces or potentially eliminates the need to endanger human workers by requiring them to access these locations for manual image capture. The drone 4 can also provide maneuverability that would otherwise be unavailable to equipment used by workers to inspect the substrate 16, thereby improving accessibility and tape inspection capabilities for these areas of the substrate 16.

[0066] Figure 8 is a conceptual diagram illustrating an embodiment of the drone-hosted substrate inspection of the present disclosure. Based on navigation commands received from the drone controller 6 or computing system 8, the control logic of the drone 4 can cause the motion guide to navigate the drone 4 to an area adjacent to a specific area of ​​the substrate 16. When the control logic detects that the drone 4 is positioned and oriented sufficiently to allow image capture by the image capture hardware 12, it can activate the image capture hardware 12 to capture one or more images of portions of the substrate 16 that are within the field of view of the lens hardware of the image capture hardware 12.

[0067] The processing circuit of the drone 4 or computing system 8 can analyze the images(s) received from the image capture hardware 12 to detect defects in the substrate 16 by executing one or more of the trained models described above (e.g., one or more of a classification model, a detection model, or a segmentation model). As described above, in various embodiments, the trained model may be a trained deep neural network such as a trained CNN. In these and other embodiments, the trained models of the Disclosure can detect defects in the substrate 16 by applying computer vision-oriented machine learning techniques.

[0068] Figures 9A–9C are conceptual diagrams showing examples of underdrive fasteners in a substrate 16 that can be detected as substrate defects by a trained model of the present disclosure. As used herein, the term “underdrive fastener” may refer to any nail, screw, bolt, rivet, or other through-fastener that is not driven into the substrate 16 to a depth sufficient to place the fastener head 44 substantially flush with the surface of the substrate 16, or to a sufficiently uniform depth. Underdrive fasteners impair the structural integrity of the building envelope or other structure represented by the substrate 16. If tape 14 is applied over a portion of the substrate 16 surrounding and containing an underdrive fastener as shown in Figure 9A, the protrusion of the fastener head 44 may result in misapplication of the tape 14 to the substrate 16 due to tenting.

[0069] An embodiment of System 10 can capture an image of the substrate 16 shown in Figure 9A by positioning the image capture hardware 12 close enough to the substrate 16 (for example, by using the drone 4 as shown in Figure 8, or via manual positioning as described above in relation to other embodiments) and activating the image capture hardware 12 to capture an image. The processing circuit of the drone 4 or the computing system 8 can run the trained model of the present disclosure using the image in Figure 9A to detect underdrive fasteners based on image data representing the position and / or orientation of the fastener head 44. In this way, the trained model of the present disclosure can, in its execution phase, provide a model output showing the underdrive state of the fastener in Figure 9A, thereby enabling timely repair of the underdrive fastener.

[0070] For example, underdrive fasteners detected by the trained model of the present disclosure based on the position and / or orientation of the fastener head 44 can be repaired based on the model output provided by the trained model of the present disclosure before further construction-related tasks are performed on the substrate 16. In this embodiment, the trained model of the present disclosure reduces or potentially eliminates the need for additional dismantling or disassembly simply to access the underdrive fasteners before repair. Instead, by detecting underdrive fasteners based on analyzing image data representing the fastener head 44 during the outer layer inspection, the trained model of the present disclosure enables the repair of underdrive fasteners in a timely and efficient manner.

[0071] The image shown in Figure 9B illustrates the effectiveness of the tape 14 when applied over an underdrive fastener defect on the substrate 16, indicated by the protrusion of the fastener head 44 in Figure 9A. In embodiments in which the trained model of this disclosure is performed using the image in Figure 9B, the model output enables various types of repairs, such as a sequence of tape 14 removal, repair of the underdrive fastener as evidenced by the position and / or orientation of the fastener head 44, and reapplication of a new segment of tape 14 to the repaired substrate 16. In some embodiments, such as embodiments in which the trained model is also trained to detect defects in the application of tape 14 to the substrate 16 (as described above with respect to Figures 1 to 6), the trained model may also communicate model output indicating misapplication of tape 14 (in this particular embodiment, tearing) at the location of the fastener head 44.

[0072] The image in Figure 9C illustrates the ineffectiveness of tape 14 when applied over an underdrive fastener defect in the substrate 16, indicated by tenting 47 of tape 14 applied to the substrate 16. Tenting 47 occurs because, even when tape 14 is applied over an underdrive fastener that is improperly embedded in the substrate 16, there is no tension in the underdrive fastener to break or penetrate tape 14. In embodiments in which a trained model of the present disclosure is performed using the image in Figure 9C, the model output enables various types of repairs, such as a sequence of removing tape 14, repairing the tenting 47 by removing the underdrive fastener or driving the underdrive fastener so that it is coplanar with the substrate 16, and reapplying a new segment of tape 14 to the repaired substrate 16 so that the new segment of tape 14 is coplanar with the substrate 16.

[0073] Figures 10A and 10B are conceptual diagrams illustrating examples of board separation in a substrate 16 that can be detected as a substrate defect by a trained model of the present disclosure. As used herein, the term “separation” refers to a non-coplanar joint (such as a non-coplanar joint 45) between two adjacent boards of the substrate 16. A non-coplanar joint 45 can represent a gap between boards that are not sufficiently tightly joined together, or a difference in slope between adjacent boards positioned at different depths, or a combination of these defects. Board separation resulting from conditions such as a non-coplanar joint 45 impairs the structural integrity of the building envelope or other structure represented by the substrate 16.

[0074] Embodiments of System 10 can capture an image of the substrate 16 shown in Figure 10A by activating the image capture hardware 12 to capture an image, based on positioning the image capture hardware 12 close enough to the substrate 16 (for example, by using the drone 4 as shown in Figure 8, or via manual positioning as described above in relation to other embodiments). The processing circuit of the drone 4 or computing system 8 can use the image in Figure 10A to run one or more of the trained models of the present disclosure to detect the presence of non-coplanar joints 45. In this way, the trained models of the present disclosure can, at their execution stage, provide a model output indicating the presence of non-coplanar joints 45, enabling timely repair of the resulting board separation. In various embodiments, the model output of the trained models of the present disclosure can enable various types of repair, such as manual repair, automated repair (for example, using a drone or other equipment), or any other suitable repair scheme or mechanism.

[0075] For example, board separation caused by non-coplanar joints 45 can be repaired based on model output provided by the trained model(s) of the present disclosure before further construction-related tasks are performed on the substrate 16. In this embodiment, the trained model of the present disclosure reduces or potentially eliminates the need for additional dismantling or disassembly simply to access the non-coplanar joints 45 before repair. Instead, by detecting board separation caused by non-coplanar joints 45 during the skin inspection, the trained model of the present disclosure enables the repair of board separation in a timely and efficient manner.

[0076] The image shown in Figure 10B illustrates the effectiveness of the tape 14 when applied over a board separation defect in the substrate 16, indicated by the non-coplanar joint 45 in Figure 10A. In embodiments in which the trained model of the present disclosure is performed using the image in Figure 10B, the model output enables various types of repairs, such as a sequence of removing the tape 14, repairing the board separation resulting from the non-coplanar joint 45, and reapplying a new segment of the tape 14 to the repaired substrate 16. In some embodiments, such as embodiments in which the model is also trained to detect defects in the application of the tape 14 to the substrate 16 (as described above with respect to Figures 1 to 7), the trained model can also communicate model output indicating the non-adhesive portion of the tape 14 at the non-coplanar joint 45.

[0077] Figure 11 is a conceptual diagram illustrating an example of a defect in the substrate 16 caused by an overdrive fastener and detected using the trained model of this disclosure. As used herein, the term “overdrive fastener” refers to a nail, screw, rivet, bolt, or other fastener driven to an excessive depth such that the head or other type of proximal end of the fastener penetrates the substrate 16 and is now positioned beneath the substrate. Overdrive of a fastener causes a hole 50 to form on the surface of the substrate 16 (and to a certain depth below the surface). The hole 50 represents a defect that may impair the structural integrity of the substrate 16, as well as the integrity of the substrate 16 with respect to shielding the interior of the building 2 from weather conditions such as temperature, water, and other elements. The hole 50 may cause heat transfer, water ingress, or other functional impairment to the substrate 16. While the hole 50 is described herein as being caused by an overdrive fastener as an example, the hole 50 may also be caused by other factors such as wind-carried debris or a detached fastener.

[0078] The processing circuit of the drone 4 or computing system 8 can run the trained model of the present disclosure using the image of Figure 11 to detect the presence of the hole 50. In this way, the trained model(s) of the present disclosure can provide a model output indicating the presence of the hole 50 at each of its execution stages(s), enabling timely repair of the resulting board separation. The trained model(s) of the present disclosure can also provide a documented trail for construction site managers, inspectors, contractors, etc., thereby assisting construction management by providing insurance-related information and clarifying disputed items in potentially future disputes. In this embodiment, the trained model of the present disclosure reduces or potentially eliminates the need for additional dismantling or disassembly simply to access the hole 50 before repair. Instead, by detecting the structural defect of the substrate 16 represented by the hole 50 during the outer layer inspection, the trained model of the present disclosure enables timely and efficient repair of the hole 50.

[0079] Figures 12A and 12B are conceptual diagrams illustrating examples of impact-related damage in the substrate 16 that can be detected as substrate defects by the trained model of this disclosure. As used herein, the term “impact-related damage” refers to any type of damage to the substrate 16 that may result from impact (inward pressure) or gouging (outward pressure). The substrate 16 may exhibit impact-related damage resulting from a number of different causes, but the examples in Figures 12A and 12B are described herein with respect to damage caused by hammer claw end gouging and hammer impact, as non-limiting examples. The impact-related damage shown in Figures 12A and 12B impairs the structural integrity of the building envelope or other structure represented by the substrate 16.

[0080] Embodiments of system 10 can capture an image of the substrate 16 shown in Figure 12A, based on activating the image capture hardware 12 to capture an image, by positioning the image capture hardware 12 close enough to the substrate 16 (for example, by using the drone 4 as shown in Figure 8, or via manual positioning as described above in relation to other embodiments). The processing circuit of the drone 4 or computing system 8 can run the trained model of the present disclosure using the image in Figure 12A to detect the presence of cracks 46 on the surface. The trained model of the present disclosure can detect cracks of varying widths and severity during its execution phase. As shown in Figure 12A, the trained model of the present disclosure detects two relatively large cracks, as well as a number of smaller cracks or "dents," on the substrate 16. In this way, the trained model of the present disclosure can provide a model output during its execution phase indicating the presence of cracks 46 on the surface, enabling timely repair of the cracks 46 on the surface.

[0081] For example, board separation caused by non-coplanar joints 45 can be repaired based on model output provided by the trained model of the present disclosure before further construction-related tasks are performed on the substrate 16. In this embodiment, the trained model of the present disclosure reduces or potentially eliminates the need for additional dismantling or disassembly simply to access the non-coplanar joints 45 before repair. Instead, by detecting board separation caused by non-coplanar joints 45 during the skin layer inspection, the trained model of the present disclosure enables the repair of board separation in a timely and efficient manner.

[0082] The image shown in Figure 12B illustrates surface indentations 48 of the substrate 16. Surface indentations 48 can be caused by excessive force and / or an inappropriate angle applied when striking the surface of the substrate 16 with a hammer, or by other factors. In embodiments in which the trained model of this disclosure is performed using the image in Figure 12B, each model output identifies a surface indentation 48, which illustrates an example of a hammer striking the exposed material (e.g., wood) located beneath a weather-resistant coating applied to the substrate 16.

[0083] Figures 13A and 13B show embodiments of the present disclosure in which the drone 4 is equipped and configured to mark potential objects of investigation (OoPSI) on a substrate 16 or tape 14. In some embodiments, embodiments of the system 10 can navigate the drone 4 to an area near an OoPSI identified using the trained model described above with respect to Figures 1 to 12B, or otherwise identified. In the embodiment of Figure 13A, the drone 4 comprises an upper mount 52. The upper mount 52 can represent any hardware or combination of hardware components that, when physically coupled to the upper surface of the drone 4 (oriented during flight), allows for further coupling of the drone 4 to additional attachments and components. In other embodiments, the drone 4 may comprise a bottom mount that allows for coupling of additional attachments and / or components via the surface of the drone 4 facing the ground during flight.

[0084] The drone 4 includes an impact-absorbing subassembly 54. In the embodiment shown in Figure 13A, the upper mount 52 connects the drone 4 to the impact-absorbing subassembly 54. In some embodiments, the impact-absorbing subassembly 54 represents a set of compression springs, which may include a single compression spring or multiple compression springs. In other embodiments, the impact-absorbing subassembly 54 may represent other types of impact absorption technology, such as hydraulic devices, compression bladder, struts, or magnetic fluid. In any case, the impact-absorbing subassembly 54 is configured to absorb and / or attenuate impact impulses by converting collision-related impacts into another form of energy that can be dissipated, such as thermal energy.

[0085] The drone 4 also includes a marking device 56. The marking device 56 can represent various types of equipment configured to mark an area of ​​the substrate 16 or an area of ​​tape 14 applied to the substrate 16. In one embodiment, the marking device 54 represents an ink supply system such as a pen, felt-tip pen, marker, or bingo bar, configured to supply ink when the distal tip of the marking device 56 makes contact with a receptacle such as the substrate 16 or tape 14 applied to the substrate 16. In another embodiment, the marking device 56 is configured to supply a self-tack paper strip onto a receptacle (e.g., the substrate 16 or tape 14 applied to the substrate 16) to which the distal tip of the marking device 56 makes contact. In yet another embodiment, the marking device 56 is configured to mark the receptacle (such as the substrate 16 or tape 14 applied to the substrate 16) in another way.

[0086] Figure 13B shows further details of a specific embodiment of the drone 4 configured in the example in Figure 143. Figure 13B is a side view of the various components coupled to the drone 4 via the upper mount 52. Figure 13B shows the compression range 58 of the impact-absorbing subassembly 54. The compression range 58 represents the length that allows the impact-absorbing subassembly 54 to temporarily reduce the overall length of the combination of components coupled to the drone 4 via the upper mount 52.

[0087] It will be understood that the compression range 58 does not represent the length of compression by the impact-absorbing subassembly 54 each time the marking device 56 collides with an object such as the substrate 16. Rather, the compression range 58 represents the maximum compression provided by the impact-absorbing subassembly 54 when the distal tip of the marking device 56 comes into contact with a rigid or semi-rigid body (e.g., the substrate 16 or the tape 14 applied to the substrate 16). In the embodiment oriented as shown in Figure 13B, the right end of the marking device 56 includes a distal tip that comes into contact with the substrate 16 as part of the OoPSI marking function described herein.

[0088] Depending on the impact force, the impact-absorbing subassembly 54 can be compressed to either the full size of the compression range 58 or less than the size of the compression range 58. In the configuration shown in Figure 13B, the impact-absorbing subassembly 54 is positioned between the marking mount 60 and the rear fastener 64. The marking mount 60 represents a component configured to receive a marking device 56. In some embodiments, the marking mount 60 has an expandable or configurable diameter and / or shape, thereby allowing the marking mount 60 to receive marking devices or other peripherals of various shapes, sizes, and form factors.

[0089] In this way, the marking mount 60 enables the use of various types of marking peripherals by the system and technology of this disclosure. The rear stopper 64 represents a rigid component having a fixed position. The rear stopper 64 enables the drone 4 to provide a reaction force against collisions between the distal tip marking device 56 and the substrate 16 or tape 14, while also responding to the compression provided by the impact-absorbing subassembly 54 up to the maximum length represented by the entire length of the compression range 58.

[0090] According to the configuration shown in Figure 13B, the drone 4 also includes a motion guide 66. In various embodiments, the motion guide 66 is a linear motion guide that provides a sliding framework for the reciprocating motion of the marking mount 60 (which holds the marking device 56) in response to the distal tip of the marking device 56 striking the substrate 16. The motion guide 66 is coupled to the drone 4 via an upper mount 52 and holds an impact-absorbing subassembly 54 in a fixed position between the motion guide 66 and the marking mount 60 using one or more fasteners (for example, in a slotted channel or another type of channel).

[0091] The control circuit of drone 4 is configured to navigate drone 4 to an area associated with an identified OoPSI (e.g., located in, containing, or adjacent to the identified OoPSI). The control circuit can achieve these movements of drone 4 using the drone 4's local position tracker and other hardware. For example, the control circuit can navigate drone 4 to an area associated with an identified OoPSI based on instructions received from the control logic of drone 4. Then, the control logic of drone 4 can navigate drone 4 to an area associated with an OoPSI based on navigation instructions received by the control logic from the processing circuit of drone 4 or computing system 8.

[0092] In some embodiments, the drone 4 can navigate to and mark OoPSIs associated with defects in the substrate 16, such as those shown in Figures 8, 9A, 10A, and 11-12B, and described in relation to those figures, as configured in the embodiments of Figures 13A and 13B. Examples of substrate defect OoPSIs that the drone 4 can navigate to and mark according to the embodiments of Figures 13A and 13B include surface cracks, underdrive fasteners, overdrive fasteners, surface gouging, excess sealant, board separation, and gaps.

[0093] In some embodiments, the drone 4 can navigate to and mark OoPSIs associated with tape misapplication(s) of tape 14 applied to a substrate 16, such as those shown in and described in Figures 2-7, 9B, and 10B, as configured in the embodiments of Figures 13A and 13B. Examples of tape misapplication-related OoPSIs that the drone 4 can navigate to and mark according to the embodiments of Figures 13A and 13B include fishmouth wrinkles, tenting of tape 14 applied to a substrate 16, missing segments(s), various types of poor adhesion, and poor tension.

[0094] Figures 14A to 14C are conceptual diagrams showing embodiments of the present disclosure in which a drone 4 is equipped and configured to repair OoPSI on a substrate 16 or tape 14 applied to the substrate 16. In some embodiments, embodiments of system 10 can navigate the drone 4 to an area near OoPSI identified using the trained model described above with respect to Figures 1 to 12B, or otherwise identified. In the embodiment of Figure 14A, the drone 4 comprises an upper mount 52 and a lower mount 68. The lower mount 68 can represent any hardware or combination of hardware components that, when physically coupled to the underside of the drone 4 or a ground-facing surface (in the orientation of flight), allow for further coupling of the drone 4 to additional attachments and components.

[0095] The drone 4 includes a dispenser subassembly 72 via a lower mount 68. The dispenser subassembly 72 includes a housing 75 for receiving a syringe 76. As shown, the housing 75 is configured to receive the syringe 76 in a position and orientation such that the applicator of the syringe 76 is positioned distal to the housing 75. Thus, the dispenser subassembly 72 is configured to house the syringe 76 in a position and orientation that allows any contents of the syringe 76 to be pushed distally from the body of the drone 4.

[0096] As explained with respect to Figure 13B, the control circuit of drone 4 is configured to navigate drone 4 to an area associated with an identified OoPSI (e.g., located in, containing, or adjacent to an identified OoPSI) based on instructions generated by the control logic of drone 4 based on navigation instructions received from the processing circuit of drone 4 or the computing system 8. The control logic of drone 4 can also receive extrusion instructions from the processing circuit of drone 4 or the computing system 8.

[0097] Figure 14B shows a top view of the dispenser subassembly 72. Based on the extrusion command received from the processing circuit, the control logic can actuate the actuator motor 77. For example, based on the received extrusion command, the control logic of the drone 4 can cause the actuator motor 77 to move the actuator arm 80 into the extension phase of the reciprocating motion. The extension phase of the reciprocating motion represents the phase in which the actuator arm 80 moves along a linear path distal to the body of the drone 4. The specified distance 82 represents the distance the actuator can move within the dispenser subassembly 72, which can correlate with the consumption of material in this form factor.

[0098] By moving the actuator arm 80 into the extension phase of its reciprocating motion, the actuator motor 77 causes the actuator arm 80 to extrude a portion of the contents of the syringe 76. Based on the fact that the drone 4 is positioned in an area associated with an identified OoPSI, the actuator motor 77 causes the actuator arm 80 to extrude the contents of the syringe 76 into the area associated with the OoPSI. In some embodiments, based on navigation commands and / or extrusion commands, the control logic of the drone 4 is configured to move the drone 4 parallel to, or substantially parallel to, the surface of the substrate 16 while the actuator arm 80 is in the extension phase of its reciprocating motion to extrude the contents of the syringe 76.

[0099] As used herein, the movement of the drone 4 substantially parallel to the surface of the substrate 16 refers to movement in any pattern substantially parallel to the XY plane of the substrate 16. By moving the drone 4 substantially parallel to the XY plane of the substrate 16 while the actuator arm 80 is in the extension phase, the control logic of the drone 4 processes navigation and extrusion commands to extrude the contents of the syringe 76 onto part, most, or all of the identified OoPSI.

[0100] In other words, the navigation command can correspond to a motion pattern that, upon completion, covers part, most, or all of the identified OoPSI. In some embodiments, based on the extrusion increment associated with the extrusion command, the control logic of the drone 4 can cause the actuator motor 77 to move the actuator arm 80 into the retraction phase of its reciprocating motion to stop the extrusion of the contents of the syringe 76. That is, the extrusion increment can define the amount of contents of the syringe 76 to be extruded to repair the OoPSI, assuming the movement of the drone 4 to cover a sufficient area of ​​the OoPSI while the contents of the syringe 76 are being extruded.

[0101] The actuator coupler 74 physically connects the distal end of the actuator arm 80 (relative to the body of the drone 4) to the proximal end of the syringe 76 (relative to the body of the drone 4), causing the proximal end of the syringe 76 to track both the extension and retraction phases of the reciprocating motion of the actuator arm 80.

[0102] Figure 14C shows further details of the slotted channel 70 shown in Figure 14A. As shown in Figure 14A, the slotted channel 70 is configured to connect the dispenser subassembly 72 to the airframe of the drone 4. The slotted channel 70 provides a self-weighted uncontrolled degree of freedom (DOF) 88 of the radial movement of the dispenser subassembly 72 with respect to a reference point of fixed points on the airframe of the drone 4. By providing an uncontrolled DOF 88 of the movement of the dispenser subassembly 72, the slotted channel 70 provides an error buffer for the radial movement of the dispenser subassembly 72 (e.g., headwind gusts, rotor cleaning, etc.).

[0103] The uncontrolled DOF 88 provided by the slotted channel 70 reduces the need for additional motor and mounted component infrastructure, which would be required in the case of a controlled DOF implementation and would thus add weight to a potentially weight-sensitive system. However, it will be understood that the controlled DOF implementation and / or a rigidly fixed implementation also coincide with the implementation of the adhesive-feeding drone of this disclosure. Figure 14C shows the pivot hub 84 and radial fasteners 86A and 86B. The radial fasteners 86A and 86B are positioned equidistant from the pivot hub 84 and provide arcs included in the uncontrolled DOF 88.

[0104] For the sake of illustration only, only the arc represented by the circumferential movement between radial fasteners 86A and 86B is shown in Figure 14C, but it will be understood that the slotted channel 70 can contain various numbers of radial fasteners to provide an uncontrolled DOF 88. In various embodiments, the syringe 76 can be loaded with various types of adhesive contents such as caulking material, general-purpose silicone adhesive, nitrocellulose adhesive, paste sealant, epoxy acrylic, or other adhesives suitable for being supplied using a dispenser subassembly 72. In some embodiments, the drone 4 may be equipped with replaceable syringes, with the syringe 76 representing the syringe currently in use and other backup and / or used syringes also being loaded.

[0105] Embodiments of the drone 4 shown in Figures 14A to 14C can supply adhesive contents from a syringe 76 to repair defects in the substrate 16, such as surface cracks, underdrive fasteners, overdrive fasteners, surface gouging, gaps or other discontinuities between boards, collision-related damage, and / or improper application of tape 14, including but not limited to fishmouth wrinkles, cracks or abrasions, wrinkles, tenting, missing tape segments, insufficient adhesion, and insufficient tension.

[0106] Figure 15 is a conceptual diagram showing another embodiment according to an aspect of the present disclosure in which a drone 4 is equipped and configured to repair OoPSI on a substrate 16 or tape 14 applied to the substrate 16. In some embodiments, an aspect of the system 10 may allow the drone 4 to navigate to an area near OoPSI identified using the trained model described above with respect to Figures 1 to 12B, or otherwise identified. In the embodiment of Figure 15, the drone 4 comprises a dispenser subassembly 90. The dispenser subassembly 90 includes a housing 94 that receives an aerosol supply system 102. Although the dispenser subassembly 90 is shown in Figure 15 as being mounted on the top surface of the drone 4 in a non-limiting embodiment, it will be understood that in other embodiments according to the present disclosure, the dispenser subassembly 90 may be coupled to the drone 4 in other ways.

[0107] The aerosol supply system 102 can represent one or more types of cans or storage devices configured to release compressed contents when a pressure valve is opened, such as by pressing down a nozzle 104. As described with respect to Figure 13B, the control circuit of the drone 4 is configured to navigate the drone 4 to an area associated with an identified OoPSI (e.g., located in, containing, or adjacent to an identified OoPSI) based on instructions generated by the control logic of the drone 4 based on navigation instructions received from the processing circuit of the drone 4 or the computing system 8. The control logic of the drone 4 can also receive supply instructions from the processing circuit of the drone 4 or the computing system 8.

[0108] Based on the supply command received from the processing circuit, the control logic can activate the motor 92. For example, based on the received supply command, the control logic of the drone 4 can cause the motor 92 to move the trigger 98 to the reversal phase of the reciprocating motion. The reversal phase of the reciprocating motion represents the phase in which the trigger 98 moves closer to the body of the drone 4. For example, when activated in this way by the control logic of the drone 4, the motor 92 can retract the link wire 96 toward the body of the drone 4, thereby retracting the trigger 98 coupled to the link wire 96.

[0109] By moving the trigger 98 during the reciprocating phase, the motor 92 causes the trigger 98 to press down on the nozzle 104, thereby releasing a portion of the contents of the aerosol supply system 102. Based on the fact that the drone 4 is positioned in an area associated with an identified OoPSI, the motor 92 causes the trigger 98 to press down on the nozzle 104, thereby supplying the contents of the aerosol supply system 102 to the area associated with the OoPSI. In some embodiments, based on navigation and / or supply commands, the control logic of the drone 4 is configured to move the drone 4 parallel to, or substantially parallel to, the surface of the substrate 16 while the trigger 98 is in the reciprocating phase, pressing down on and holding down the nozzle 98, thereby supplying the contents of the aerosol supply system 102.

[0110] As used herein, the movement of the drone 4 substantially parallel to the surface of the substrate 16 refers to movement in any pattern substantially parallel to the XY plane of the substrate 16. By moving the drone 4 substantially parallel to the XY plane of the substrate 16 while the trigger 98 is in the retraction phase of the reciprocating motion, the control logic of the drone 4 processes navigation and extrusion commands to deliver the contents of the aerosol supply system 102 onto some, most, or all of the identified OoPSI.

[0111] In other words, the navigation command can correspond to a motion pattern that, upon completion, covers part, most, or all of the identified OoPSI. In some embodiments, based on the feed increment associated with the extrusion command, the control logic of the drone 4 can cause the motor 92 to release at least part of the tension applied to the link wire 96, thereby triggering the trigger 98 during the extension phase of the reciprocating motion and stopping the supply of contents from the aerosol supply system 102. That is, the feed increment can define the amount of contents from the aerosol supply system 102 sprayed to repair the OoPSI, assuming the drone 4 moves to cover a sufficient area of ​​the OoPSI while the contents of the aerosol supply system 102 are being sprayed.

[0112] The contents of the aerosol supply system 102 may include any aerosol propulsion sealant, or any other material suitable for spraying onto an OoPSI identified for sealing or molding purposes, such as a rubber sealant, weather-resistant spray paint, or pressure foam sealant. The embodiment of the drone 4 shown in Figure 15 can supply the contents of the aerosol supply system 102 to repair defects in the substrate 16, such as surface cracks, overdrive fasteners, surface gouging, gaps or other discontinuities between boards, collision-related damage, and / or improper application of tape 14, such as cracks or scratches, missing tape segments, or insufficient adhesion.

[0113] In several implementations consistent with Figures 14A to 15, the drone 4 may comprise a light source, a photosensor, and an optical fiber link coupling the light source to the photosensor. In these implementations, the control logic of the drone 4 can activate the light source based on a supply / extrude command, and a motor 92 or actuator motor 77 (optionally) is configured to move a trigger 98 during the retraction phase of each reciprocating motion, or to move an actuator arm 80 during the extension phase. In these embodiments, the control logic of the drone 4 uses these light-based techniques to supply the contents of the aerosol supply system 102 or syringe 76 to the area associated with the OoPSI by pushing down the nozzle 104 or extruding the contents of the syringe 76 in response to the photosensor detecting the activation of the light source via the optical fiber link.

[0114] In other implementations consistent with Figures 14A to 15, the drone 4 may comprise a microcontroller, Bluetooth® or other short-range, low-power, wireless transceiver, and a power source such as a battery or battery pack. The microcontroller can execute a script sequentially, which may, in appropriate function calls, initiate a connection with the wireless transceiver and transmit signals corresponding to supply increments or extrusion increments. In these embodiments, the microcontroller-transceiver-based subsystem is separate and independent of the drone 4's firmware and is therefore portable across different underlying UAV platforms, with certain mechanical adjustments reserved for adaptation to the underlying UAV platform, and independent of different underlying UAV platforms.

[0115] Figure 16 is a flowchart illustrating an exemplary process 110 of the present disclosure. Process 110 can begin with an inspection of building 2 (106). For example, the control logic of drone 4 can navigate drone 4 and activate image capture hardware 12 to capture one or more images of building 2. Next, processing circuits of drone 4 or computing system 8 can analyze one or more images (108). In various embodiments, processing circuits of drone 4 or computing system 8 can analyze the images(s) received from the image capture hardware by running one or more of the trained classification models, trained detection models, or trained segmentation models of the present disclosure to generate a model output.

[0116] The processing circuit may report a model output (112). For example, the processing circuit may be communicatively coupled to output hardware which is communicatively coupled to the processing circuit. In these embodiments, the processing circuit may be configured to output a model output via output hardware, which may be a monitor, a speaker, a communication interface configured to relay the model input to another device, etc. As described above with respect to Figures 4A-4D and Figures 6A-6D, the model output may indicate a specific OoPSI(s)(s) shown in defect conditions and / or images(s)(s).

[0117] Process 110 includes a determination (decision block 114) of whether to mark the detected OoPSI using the drone 4. If the determination is to mark the detected OoPSI using the drone 4 ("yes" branch of decision block 114), the control logic of the drone 4 causes the drone 4 to mark the OoPSI, for example, by using the techniques described above with respect to 13A and 13B (116). If the determination is not to mark the detected OoPSI using the drone 4 ("no" branch of decision block 114), the field manager can optionally manually mark the detected OoPSI (118). The optional nature of manually marking the detected OoPSI is indicated by the dashed boundary in step 118 of Figure 16.

[0118] Process 110 also includes a determination (decision block 120) of whether to use drone 4 to repair the detected OoPSI. If the determination is to use drone 4 to repair the detected OoPSI ("yes" branch of decision block 120), the control logic of drone 4 can cause drone 4 to repair the OoPSI, for example, by using the techniques described above with respect to 14A-15 (122). If the determination is not to use drone 4 to repair the detected OoPSI ("no" branch of decision block 120), the field manager can optionally repair the detected OoPSI manually (124). The optional nature of manual repair of the detected OoPSI is indicated by the dashed boundary of step 124 in Figure 16. In various embodiments, the control logic of drone 4 can be configured to navigate drone 4 to the area surrounding the OoPSI and achieve remediation measures, depending on whether the processing circuit has detected marks placed manually or by drone 4 by analyzing images (one or more) received from image capture hardware 12.

[0119] In some implementations, a software application running on a computing system 8 (which is communicatively coupled to a controller 6 in these implementations) autonomously identifies one or more targets on a substrate 16 that will be repaired via spraying by an aerosol supply system 102. The application can process video data from a video feed received from the drone 4 (e.g., via image capture hardware 12 or other video capture hardware that the drone 4 may have). For example, the application can identify a crack between two plywood boards and instruct the control logic of the drone 4 to align the drone 4 with the edge or end of the crack, activate the aerosol dispenser system 90 to start spraying, and move the drone 4 along the crack until it reaches the opposite end of the crack, at which point the control logic can stop the aerosol supply system 102 and stop spraying.

[0120] In another embodiment, the application can identify a gap surrounding the joint between the pipe and the substrate 16, instruct the control logic of the drone 4 to align the drone 4 with the edge of the crack, activate the aerosol dispenser system 90 to begin spraying, and move the drone 4 along a circular path tracking the joint between the pipe and the substrate 16 until the drone 4 has completely circled the joint, at which point the control logic can stop the aerosol supply system 102 and stop spraying. In any of these embodiments, the application can identify cracks, pipes, or pipe joints by running a computer vision-oriented machine learning model trained on a dataset of numerous images of the substrate 16 at different distances, angles, lighting conditions, etc. Although described as “circular,” it will be understood that the path of the drone 4 for repairing the gap in the pipe joint may be oval, elliptical, or any other type of closed shape.

[0121] Computer vision processing can be performed in an area within a labeled bounding box around the area of ​​interest. In one embodiment of the computer vision processing workflow of this disclosure, an application running on a computing system 8 can run a trained machine learning algorithm to read video frames received from image capture hardware 12, and can separate the object of interest from the background of the image (e.g., using color masking or other techniques), refine the mask (e.g., using morphological operations such as expansion and erosion), and detect one or more edges (e.g., using Canny edge detection).

[0122] In this embodiment, the trained machine learning algorithm can erode the mask, remove outer edges, fit lines to edges (e.g., using Huff transform), filter out less relevant or irrelevant Huff lines (e.g., using DBSCAN clustering), and refine the intersections between Huff lines and mask edges(one or more). In this embodiment, the trained machine learning algorithm can find the best-fitting intersection (e.g., using k-means clustering), calculate the distance from the best-fitting interaction point to the video center, and pass variables to the control logic of the drone 4 via a wireless communication connection.

[0123] This variable can represent the crack initiation point, the crack, the angle, and other parameters that enable the control logic to navigate the drone 4 in a manner that allows the aerosol supply system 102 to completely repair the detected crack(s) or crack(s). Although the use of the aerosol supply system 102 is described primarily, it will be understood that aspects of the computer vision processing of this disclosure also enable OoPSI marking (e.g., using the configurations shown in Figures 13A-13C) and / or OoPSI repair using adhesive supply as shown by the embodiments of Figures 14A and 14B.

[0124] An exemplary procedure by which an embodiment of System 10 performs a repair procedure (whether using the dispenser subassembly 72 or the aerosol supply system 102) is described below. In this embodiment, the control logic of the drone 4 can align the drone 4 with the OoPSI to be repaired. The control logic can then activate either the dispenser subassembly 72 or the aerosol supply system 102 using any mechanism consistent with the present disclosure, such as the optical toggle mechanism or the microcontroller-based mechanism described above, and an embodiment of System 10 can perform the computer vision procedure described above.

[0125] Based on the output of the computer vision procedure, the processing circuit may determine whether the OoPSI angle (e.g., crack angle) is within a predetermined range. If the crack angle is not within the predetermined range, the control logic may adjust the yaw of the drone 4 relative to the substrate 16 and re-execute the computer vision procedure to evaluate the OoPSI angle.

[0126] If the OoPSI is within a predetermined range (whether in the first or subsequent iteration of the angular evaluation with respect to the execution path of the computer vision procedure), the processing circuit can determine whether the edge of the OoPSI (e.g., the edge of a crack) is at the center or substantially at the center of a video frame or other image captured by the image capture hardware 12. If the edge of the OoPSI is not at the center of the frame or is not substantially at the center, the control logic can adjust the pitch and / or roll of the drone 4 to move along the OoPSI, thereby aligning either the dispenser subassembly 72 or the aerosol supply system 102 with the edge of the OoPSI and initiating repair at the appropriate location.

[0127] The processing circuit may iteratively re-execute the computer vision procedure until the edge of the OoPSI is substantially centered in the most recently captured frame via the image capture hardware. During the execution cycle of the computer vision procedure, which indicates that the OoPSI angle is within a predetermined range and the edge of the OoPSI is substantially centered in the frame captured by the image capture hardware 12, the control logic may stop the dispenser subassembly 72 or the aerosol supply system 102 to repair the OoPSI (e.g., using any of the optical toggle mechanism described above, the microcontroller-based mechanism described above, or any other operating mechanism consistent with the present disclosure).

[0128] In the following detailed description of preferred embodiments, reference is made to accompanying drawings illustrating specific embodiments by which the present invention can be carried out. The illustrated embodiments are not intended to exhaust all embodiments of the present invention. It should be understood that other embodiments may be utilized, and structural or logical modifications may be made without departing from the scope of the invention. Accordingly, the following detailed description should not be construed as restrictive, and the scope of the invention is defined by the appended claims.

[0129] Unless otherwise indicated, all numbers used in this specification and the claims to describe the size, quantity, and physical properties of feature parts should be understood in all cases as being modified by the terms “about,” “approximately,” or “substantially.” Therefore, unless otherwise indicated, the numerical parameters described in the above specification and the appended claims are approximations that may vary depending on the desired properties sought by those skilled in the art using the teachings disclosed herein.

[0130] As used herein and in the appended claims, the singular forms "a," "an," and "the" include embodiments having multiple references unless otherwise explicitly stated. As used herein and in the appended claims, the term "or" generally includes "and / or" unless otherwise explicitly stated.

[0131] Examples demonstrate that any particular action or event of the methods described herein can be performed in a different order, and can be added, combined, or omitted entirely (for example, not all of the described actions or events are necessary for the implementation of the method). Furthermore, in certain embodiments, the actions or events can be performed not sequentially, but concurrently, for example, through multithreading, interrupt handling, or by multiple processors.

[0132] The technologies described herein may be implemented at least partially in hardware, software, firmware, or any combination thereof. For example, various aspects of the technologies described may be implemented in one or more processors, including one or more microprocessors, CPUs, GPUs, DSPs, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or any other equivalent integrated circuits or discrete logic circuits, and any combination of such components. The terms “processor” or “processing circuit” may generally refer to the aforementioned logic circuits alone or in combination with other logic circuits, or other equivalent circuits. A control unit including hardware may also perform one or more of the technologies described herein.

[0133] Such hardware, software, and firmware may be implemented in the same device or in separate devices to support the various operations and functions described herein. Furthermore, any of the described units, modules, or components may be implemented together or separately as individual but interoperable logical devices. The descriptions of different features of a module or unit are intended to highlight different functional aspects and do not necessarily imply that such a module or unit must be implemented by separate hardware or software components. Rather, the functions associated with one or more modules or units may be performed by separate hardware or software components or incorporated into common or separate hardware or software components.

[0134] The techniques described herein may also be embodied or encoded in a computer-readable medium, such as a computer-readable storage medium containing instructions. Instructions embedded or encoded in a computer-readable storage medium may, for example, cause a programmable processor or other processor to execute a method when the instructions are executed. Examples of computer-readable storage media include random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM), flash memory, hard disks, CD-ROMs, floppy disks, cassettes, magnetic media, optical media, or other computer-readable media.

[0135] Various embodiments have been described. These and other embodiments are within the scope of the following claims. In addition to each embodiment, the following embodiments are also described. (Note 1) Polarization camera hardware configured to capture a polarized image of a tape applied to a substrate, A memory communicatively coupled to the polarization camera hardware, the memory configured to store the polarization image, A processing circuit that is communicatively coupled to the memory, The polarization image is analyzed according to a trained classification model. Based on the analysis of the polarized image according to the trained classification model, misapplication of the tape applied to the substrate is detected. A processing circuit configured as follows, A system equipped with these features. (Note 2) The system described in Appendix 1, wherein the aforementioned trained classification model is a trained neural network model. (Note 3) The system described in Appendix 1, wherein the polarization camera hardware is incorporated into a mobile computing device. (Note 4) The system as described in Appendix 1, wherein the polarization camera hardware is communicatively coupled to a mobile computing device. (Note 5) The system described in Appendix 3, wherein the mobile computing device includes one of a smartphone, a tablet computer, or a wearable computing device. (Note 6) The system as described in Appendix 1, wherein the polarization camera hardware is communicatively coupled to the drone. (Note 7) The system described in Appendix 1, wherein the aforementioned polarizing camera hardware is incorporated into the drone. (Note 8) The system according to Appendix 1, further comprising output hardware communicatively coupled to the processing circuit, wherein the processing circuit is further configured to output a model output indicating the misapplication of the tape applied to the substrate via the output hardware. (Note 9) The system according to Appendix 1, wherein the aforementioned misapplication is associated with at least one of the fishmouth wrinkles or creases of the tape applied to the substrate. (Note 10) The system as described in Appendix 1, wherein the trained classification model is configured to implement one or more of the following with respect to the image: whole image classification, sub-image classification, object detection, or image segmentation. (Note 11) The system as described in Appendix 1, wherein the substrate is the outer layer of a building. (Note 12) The system as described in Appendix 1, wherein the polarization image shows the division of focal plane (DoLP) data relating to the tape applied to the substrate. (Note 13) Polarization camera hardware captures the polarized image of the tape applied to the substrate, A processing circuit, which is communicatively coupled to the image capture hardware, analyzes the polarized image according to a trained classification model, The processing circuit detects misapplication of the tape applied to the substrate based on the analysis of the polarized image according to the trained classification model, Methods that include... (Note 14) The method described in Appendix 13, wherein the trained model is a trained neural network model. (Note 15) The method according to Appendix 13, wherein the polarization camera hardware is incorporated into a mobile computing device. (Note 16) The method described in Appendix 13, wherein the polarization camera hardware is incorporated into a drone. (Note 17) The method according to Appendix 13, further comprising the processing circuit outputting a model output indicating the misapplication of the tape applied to the substrate via output hardware communicably coupled to the processing circuit. (Note 18) The method according to Appendix 13, wherein the misapplication is associated with at least one of the fishmouth wrinkles or wrinkles of the tape applied to the substrate. (Note 19) The method according to Appendix 13, wherein the trained model is configured to implement one or more of the following with respect to the image: whole image classification, sub-image classification, object detection, or image segmentation. (Note 20) The method according to Appendix 13, wherein the substrate is the outer layer of a building. (Note 21) The method according to Appendix 13, wherein the polarization image shows the division of focal plane (DoLP) data relating to the tape applied to the substrate. (Note 22) A means for capturing a polarized image of a tape applied to a substrate, Means for analyzing the polarized image according to a trained model, A means for detecting misapplication of the tape applied to the substrate, based on the analysis of the polarized image according to the trained model, A device equipped with the following features. (Note 23) A computer-readable storage device in which instructions are encoded, wherein when the instructions are executed, the processing circuit of a computing device is configured to receive the instructions. The polarization image of the tape applied to the substrate is received from the polarization camera hardware. The polarization image is stored in the computer-readable storage device. The polarization image is analyzed according to the trained model, Based on the analysis of the polarized image according to the trained model, the misapplication of the tape applied to the substrate is detected. Computer-readable memory device.

Claims

1. Polarization camera hardware configured to capture multiple polarization images based on multiple polarization directions of a tape applied to a substrate, A memory communicatively coupled to the polarization camera hardware, the memory being configured to store the plurality of polarization images, A processing circuit that is communicatively coupled to the memory, The multiple polarization images are analyzed according to a trained classification model. Based on the results of the above analysis, the misapplication of the tape applied to the substrate is detected. A processing circuit configured as follows, A system equipped with these features.

2. The system according to claim 1, wherein the trained classification model is a trained neural network model.

3. The system according to claim 1, wherein the polarization camera hardware is incorporated into a mobile computing device.

4. The system according to claim 1, wherein the polarization camera hardware is communicably coupled to a mobile computing device.

5. The system according to claim 3, wherein the mobile computing device includes one of a smartphone, a tablet computer, or a wearable computing device.

6. The system according to claim 1, wherein the polarization camera hardware is incorporated into the drone.

7. The system according to claim 1, further comprising output hardware communicatively coupled to the processing circuit, wherein the processing circuit is further configured to output a model output indicating the misapplication of the tape applied to the substrate via the output hardware.

8. The system according to claim 1, wherein the misapplication is associated with at least one of fishmouth wrinkles or creases of the tape applied to the substrate.

9. The system according to claim 1, wherein the trained classification model is configured to implement one or more of the following with respect to the image: whole image classification, sub-image classification, object detection, or image segmentation.

10. The system according to claim 1, wherein the substrate is the outer layer of a building.

11. Polarization camera hardware captures multiple polarization images based on multiple polarization directions of the tape applied to the substrate, A processing circuit communicatively coupled to the polarization camera hardware analyzes the plurality of polarization images according to a trained classification model, The processing circuit detects, based on the results of the analysis, a misapplication of the tape applied to the substrate, Methods that include...

12. The method according to claim 11, wherein the trained model is a trained neural network model.

13. The method according to claim 11, wherein the polarization camera hardware is incorporated into a mobile computing device.

14. The method according to claim 11, wherein the polarization camera hardware is incorporated into the drone.

15. The method according to claim 11, further comprising the processing circuit outputting a model output indicating the misapplication of the tape applied to the substrate via output hardware communicably coupled to the processing circuit.

16. The method according to claim 11, wherein the misapplication is associated with at least one of the fishmouth wrinkles or wrinkles of the tape applied to the substrate.

17. The method according to claim 11, wherein the trained model is configured to implement one or more of the following with respect to the image: whole image classification, sub-image classification, object detection, or image segmentation.

18. The method according to claim 11, wherein the substrate is the outer layer of a building.

19. Means for capturing multiple polarization images based on multiple polarization directions of a tape applied to a substrate, Means for analyzing the plurality of polarized images according to a trained model, Based on the results of the above analysis, means for detecting misapplication of the tape applied to the substrate, A device equipped with the following features.

Images