Building inspection based on polarized images

Abstract

A system of the present disclosure enables building inspections using polarized images. The system detects misapplication of tape applied to a substrate using light polarization information of the polarized images. An exemplary system includes polarization camera hardware, a memory communicatively coupled to the polarization camera hardware, and a processing circuit communicatively coupled to the memory. The polarization camera hardware is configured to capture a polarized image of the tape applied to the substrate. The memory is configured to store the polarized image. The processing circuit is configured to analyze the polarized image according to a trained classification model and detect misapplication of the tape applied to the substrate based on the analysis of the polarized image according to the trained classification model.

Description

[Technical field]

[0001] The present disclosure relates generally to the field of construction-related functions performed using drones. [Background technology]

[0002] During or after construction, buildings are subject to inspection. If certain defects are detected during such inspection, whether during construction or at the post-completion stage, the defects can be marked for future identification and / or repaired. Summary of the Invention

[0003] The present disclosure describes a system configured for drone-based building inspection, building defect marking, and building defect repair. The present disclosure primarily describes drone-hosted techniques as being performed on a building envelope layer during construction of the building, as a non-limiting example. However, it will be appreciated that the various drone-hosted techniques of the present disclosure are applicable to various aspects of a building, whether the building is currently under construction or fully constructed. Some examples of the present disclosure leverage camera hardware integrated into a drone to capture one or more images of the building (e.g., of the building envelope). According to these examples, the system of the present disclosure uses a trained machine-learning (ML) model to analyze the image(s) to determine whether a portion of the building shown in the image(s) includes a defect that the ML model is trained to detect.

[0004] Some embodiments of the present disclosure are directed to drone-hosted marking operations for building defects. In these embodiments, the drone can include or be coupled to a marking subsystem, such as an ink supply subsystem or a self-tack paper supply subsystem. In these embodiments, the system of the present disclosure can activate the marking subsystem to mark an area of ​​the identified defect or an area near the identified defect. Some embodiments of the present disclosure are directed to drone-hosted repair operations for building defects. In these embodiments, the drone can include or be coupled to a repair subsystem, such as an aerosol supply subsystem or an adhesive supply subsystem. In these embodiments, the system of the present disclosure can activate the repair subsystem to supply aerosol or adhesive (as the case may be) to an area associated with the identified defect.

[0005] In one embodiment, the system includes polarization camera hardware, a memory communicatively coupled to the polarization camera hardware, and a processing circuit communicatively coupled to the memory. The polarization camera hardware is configured to capture a polarization image of a tape applied to a substrate. The memory is configured to store the polarization image. The processing circuit is configured to analyze the polarization image according to a trained classification model and detect misapplication related to the tape applied to the substrate based on the analysis of the polarization image according to the trained classification model.

[0006] In another embodiment, a method includes capturing, by polarization camera hardware, a polarization image of tape applied to a substrate. The method further includes analyzing, by processing circuitry communicatively coupled to the image capture hardware, the polarization image according to the trained classification model. The method further includes detecting, by the processing circuitry, a misapplication related to the tape applied to the substrate based on the analysis of the polarization image according to the trained classification model.

[0007] In another embodiment, an apparatus includes means for capturing a polarized image of tape applied to a substrate, means for analyzing the polarized image according to the trained model, and means for detecting misapplication of the tape applied to the substrate based on the analysis of the polarized image according to the trained model.

[0008] In another example, a computer readable storage device is encoded with instructions that, when executed, cause a processing circuitry of the computing device to receive a polarized image of tape applied to a substrate from the polarization camera hardware, store the polarized image in the computer readable storage device, analyze the polarized image according to the trained model, and detect misapplication of the tape applied to the substrate based on the analysis of the polarized image according to the trained model.

[0009] The disclosed system offers several potential advantages over currently available solutions: By hosting image capture, defect marking, and defect repair operations on the drone, the disclosed system improves safety and improves data accuracy by reducing the occurrence of human error when workers are deployed to field locations that may have varying weather / visibility conditions and are at high altitude locations. The disclosed defect detection technique runs a trained ML model (which in various embodiments according to the present disclosure may be a classification model, a detection model, or a segmentation model) to analyze image data of an area of ​​a building, thereby reducing the possibility of human error where safety concerns are paramount.

[0010] Additionally, the drone-hosted technology of the present disclosure can improve the accuracy and completeness of the inspection, marking, or repair by leveraging the maneuverability of the drone to more thoroughly inspect a building (or other structure) and perform the inspection, marking, or repair in areas that may be difficult for human workers to reach. In some embodiments, the drones of the present disclosure are equipped with specialized image capture hardware, thereby providing images that the trained models of the present disclosure can analyze with greater accuracy than the human eye can interpret a standard image or direct view of a building. In this manner, the drone-hosted technology of the present disclosure can improve data accuracy and / or process completeness while also providing practical applications of enhanced safety.

[0011] The details of one or more embodiments of the disclosure are set forth in the accompanying drawings and the following specification. Other features, objects, and advantages of the disclosure will become apparent from the description and drawings, and from the claims. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a conceptual diagram illustrating an embodiment of a system, aspects of which are configured to perform one or more techniques of the present disclosure.

[0013] [Diagram 2] FIG. 1 is a conceptual diagram illustrating an aspect of drone-hosted tape-applied inspection of the present disclosure.

[0014] [Figure 3A] FIG. 2 is a conceptual diagram showing further details of the misapplication of tape to a substrate that an embodiment of the system of FIG. 1 can detect using the techniques of the present disclosure. [Figure 3B] FIG. 2 is a conceptual diagram showing further details of the misapplication of tape to a substrate that an embodiment of the system of FIG. 1 can detect using the techniques of the present disclosure.

[0015] [Figure 4A]FIG. 1 illustrates various deep learning generated image labels that a trained classification model of the present disclosure can generate. [Figure 4B] FIG. 1 illustrates various deep learning generated image labels that a trained classification model of the present disclosure can generate. [Figure 4C] FIG. 1 illustrates various deep learning generated image labels that a trained classification model of the present disclosure can generate. [Figure 4D] FIG. 1 illustrates various deep learning generated image labels that a trained classification model of the present disclosure can generate.

[0016] [Diagram 5] 2 is a conceptual diagram illustrating a polarized image that the system of FIG. 1 can analyze to detect defects related to a tape applied to a substrate, according to an embodiment of the present disclosure.

[0017] [Figure 6A] FIG. 6 illustrates various deep learning generated image labels that a trained classification model of the present disclosure can generate using the polarized image shown in FIG. [Figure 6B] FIG. 6 illustrates various deep learning generated image labels that a trained classification model of the present disclosure can generate using the polarized image shown in FIG. [Figure 6C] FIG. 6 illustrates various deep learning generated image labels that a trained classification model of the present disclosure can generate using the polarized image shown in FIG. [Figure 6D] FIG. 6 illustrates various deep learning generated image labels that a trained classification model of the present disclosure can generate using the polarized image shown in FIG.

[0018] [Figure 7] 1 is a graph illustrating aspects of polarized light image analysis that a trained classification model of the present disclosure can perform to detect one or more defects related to a tape applied to a substrate.

[0019] [Figure 8]FIG. 1 is a conceptual diagram illustrating an aspect of drone-hosted substrate inspection of the present disclosure.

[0020] [Figure 9A] FIG. 13 is a conceptual diagram illustrating an example of an underdriven fastener in a substrate that a trained classification model of the present disclosure can detect as a substrate defect. [Figure 9B] FIG. 13 is a conceptual diagram illustrating an example of an underdriven fastener in a substrate that a trained classification model of the present disclosure can detect as a substrate defect. [Figure 9C] FIG. 13 is a conceptual diagram illustrating an example of an underdriven fastener in a substrate that a trained classification model of the present disclosure can detect as a substrate defect.

[0021] [Figure 10A] FIG. 1 is a conceptual diagram illustrating an example of a board separation in a substrate that a trained classification model of the present disclosure can detect as a substrate defect. [Figure 10B] FIG. 1 is a conceptual diagram illustrating an example of a board separation in a substrate that a trained classification model of the present disclosure can detect as a substrate defect.

[0022] [Figure 11] FIG. 13 is a conceptual diagram illustrating an example of a substrate defect caused by an overdriven fastener and detected using a trained classification model of the present disclosure.

[0023] [Figure 12A] FIG. 1 is a conceptual diagram illustrating an example of impact-related damage in a substrate that a trained classification model of the present disclosure can detect as a substrate defect. [Figure 12B] FIG. 1 is a conceptual diagram illustrating an example of impact-related damage in a substrate that a trained classification model of the present disclosure can detect as a substrate defect.

[0024] [Figure 13A]FIG. 1 illustrates an example in which a drone is equipped and configured to mark potential objects of interest on a substrate or tape, according to aspects of the present disclosure. [Figure 13B] FIG. 1 illustrates an example in which a drone is equipped and configured to mark potential objects of interest on a substrate or tape, according to aspects of the present disclosure.

[0025] [Figure 14A] FIG. 1 is a conceptual diagram illustrating an example in which a drone is equipped and configured to repair a potential object of interest on a substrate or a tape applied to a substrate, according to aspects of the present disclosure. [Figure 14B] FIG. 1 is a conceptual diagram illustrating an example in which a drone is equipped and configured to repair a potential object of interest on a substrate or a tape applied to a substrate, according to aspects of the present disclosure. [Figure 14C] FIG. 1 is a conceptual diagram illustrating an example in which a drone is equipped and configured to repair a potential object of interest on a substrate or a tape applied to a substrate, according to aspects of the present disclosure.

[0026] [Figure 15] FIG. 13 is a conceptual diagram illustrating another example in which a drone is equipped and configured to repair a potential object of interest on a substrate or a tape applied to a substrate, in accordance with aspects of the present disclosure.

[0027] [Figure 16] 1 is a flowchart illustrating an exemplary process of the present disclosure.

[0028] It is understood that the embodiments may be utilized and structural changes may be made without departing from the scope of the present invention. The figures are not necessarily drawn to scale. Like numbers used in the figures indicate like components. However, it will be understood that the use of a number to indicate a component in a given figure is not intended to limit the component in another figure that is indicated with the same number. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] FIG. 1 is a conceptual diagram illustrating an embodiment of a system 10, the embodiment of which is configured to perform one or more techniques of the present disclosure. The system 10 includes a building 2, a drone 4, a drone controller 6, and a computing system 8. The building 2 is shown as being in a construction stage during which the exposed layer facing the outside is the "skin layer" or "building envelope." Although the techniques of the present disclosure are described as being performed on the building envelope as a non-limiting example, it will be understood that the various techniques of the present disclosure are applicable to other substrates as well. Examples of other substrates include non-building structures such as completed building walls, walls, fences, bridges, ships, aircraft, cell phone towers, etc., whether outdoors or indoors.

[0030] Building envelope refers to the physical barrier between the conditioned and unconditioned environments of the respective building (in this case, Building 2). In various embodiments, the building envelope may be referred to as a "building enclosure," an "envelope layer" as described above, or a "weatherproof barrier" ("WPB"). The building envelope shields the interior of the building from the outdoor elements and plays a vital role in environmental control. Aspects of the elemental shielding and environmental control functions of the building envelope include rain protection, air control, heat transfer control, and vapor protection. Thus, the integrity of the building envelope is essential to the safety and habitability of Building 2.

[0031] With respect to tasks such as inspecting a building envelope for defects, marking any detected defects, or repairing any detected defects in the building envelope, accuracy and completeness are critical to the goal of maintaining integrity. As buildings increase in size, design complexity, and density, performing these tasks manually becomes increasingly difficult. The system 10 may leverage the maneuverability of a drone (e.g., drone 4) to perform one or more of the building envelope inspection, defect marking, and / or defect repair. The system 10 may also leverage dedicated computing capabilities to identify the presence of potential defects, the location of such potential defects, if any, and / or parameters of operations to be performed to repair such potential defects, if any. These dedicated computing capabilities may be provided by computing or processing hardware of one or more of the drone 4, drone controller 6, and / or computing system 8. In some embodiments, aspects of the system 10 may leverage cloud computing resources to implement the dedicated computing capabilities in a distributed manner.

[0032] Drone 4 may represent one or more types of unmanned aerial vehicles (UAVs). In various embodiments, drone 4 may also be referred to as one or more of an autonomous aerial vehicle, an autopilot vehicle, a remotely piloted aircraft, a remotely piloted aircraft, a remotely piloted aircraft system, a remotely piloted aerial system, a remotely piloted aerial vehicle, a remotely piloted system, a remotely piloted vehicle, a mini unmanned aerial vehicle system, a mini unmanned aerial vehicle, an unmanned flight system, an unmanned aerial vehicle, a remotely piloted transport system, and the like.

[0033] The processing circuitry of drone controller 6 and / or computing system 8 can formulate navigation instructions for drone 4 based on the location of an area of ​​building 2 that is subject to inspection, defect marking, or defect repair by drone 4 and its respective subsystems. The processing circuitry can then, as the case may be, activate the wireless interface hardware of drone controller 6 or computing system 8 to transmit the navigation instructions to the wireless interface hardware of drone 4. The wireless interface hardware of drone 4, drone controller 6, and computing system 8 can represent communications hardware, such as by enabling wireless communication between two or more of drone 4, drone controller 6, and / or computing system 8, and with other devices equipped with wireless interface hardware.

[0034] The drone 4 may include a motion guide that controls the movement of the drone 4, such as the flight path of the drone 4. The drone 4 may also include control logic that receives navigation instructions from either the drone controller 6 or the computing system 8 via the drone 4's wireless interface hardware. The control logic may use the navigation instructions received from the drone controller 6 or the computing system 8 to navigate the drone 4 to an area proximate to a particular portion of the building 2. In various examples consistent with aspects of the present disclosure, the processing circuitry of the drone controller 6 and / or the computing system 8 may form navigation instructions based on an area of ​​the building 2 to be inspected for objects of potential survey interest (OoPSI) or based on an area associated with a previously identified OoPSI to facilitate marking and / or repair of identified OoPSI.

[0035] Computing system 8 may include or be part of one or more of various types of computing devices, such as a mobile phone (e.g., a smartphone), a tablet computer, a netbook, a laptop computer, a desktop computer, a personal digital assistant ("PDA"), a wearable device (e.g., a smart watch or smart glasses), among others. In some examples, computing system 8 may represent a distributed system including an interconnected network of two or more such devices. Computing system 8 is shown in FIG. 1 as a laptop computer as a non-limiting example according to aspects of the present disclosure.

[0036] Drone controller 6, in many embodiments, represents a wireless control transmitter or transceiver. Drone controller 6 is configured to process user input received via various input hardware (e.g., joystick, buttons, etc.), formulate navigation instructions as described above, and transmit the navigation instructions in substantially real-time via communication interface hardware to the communication interface hardware (e.g., receiver) of drone 4. The complementary communication interfaces of drone 4 and drone controller 6 can communicate via one or more predefined frequencies.

[0037] In this manner, aspects of the system 10 leverage the flight capabilities and maneuverability of the drone 4 to inspect the building 2 and, in some scenarios,

number

[0038] FIG. 2 is a conceptual diagram illustrating an embodiment of a drone-hosted tape application inspection of the present disclosure. FIG. 2 shows a substrate 16, which in some examples may represent a portion of an exterior surface of a building 2, such as a wall (as shown), a roof, etc. In this example, the substrate 16 comprises a tape 14. The tape 14 may represent any of a variety of types of adhesive-coated materials. This disclosure primarily describes a non-limiting example in which the tape 14 represents a so-called "flashing tape" that is commonly used to seal seams, cracks, or other discontinuities in a building exterior surface, such as the substrate 16. In some examples, the substrate 16 may represent a surface coated with an adhesive layer, such as a roll-on adhesive that leaves an outwardly facing adhesive layer on the surface. Some non-limiting example features of one or more of tape 14, a sealant that substrate 16 may be coated with, and / or an outwardly facing adhesive that substrate 16 may be coated with are described in published U.S. patent applications having publication numbers US20130139953A1, US2020003098A1, and US20190031923A1, as well as publication numbers WO2021033111A1, WO2021024206A1, WO201601 No. 9248A1, WO2016106273A1, WO2015183354A2, WO2015126931A1, WO2017031275A1, W02019152621A1, WO2017112756A1, WO2017031275A1, WO2018156631A1, and WO2018220555A1, the entire disclosures of each of which are incorporated herein by reference.

[0039] 2, drone 4 includes image capture hardware 12. In some embodiments, image capture hardware 12 represents one or more types of digital cameras, such as cameras configured to store captured still images and / or video in a digital format (e.g., as a .jpeg file, a .png file, an .mp4 file, etc.). Based on navigation instructions received from drone controller 6 or computing system 8, control logic of drone 4 can cause a motion guide to navigate drone 4 to an area proximate to a particular area of ​​tape 14 applied to substrate 16. Upon detecting that drone 4 is sufficiently positioned and oriented to enable image capture by image capture hardware 12, the control logic can activate image capture hardware 12 to capture one or more images of a portion of tape 14 that is within the field of view of lens hardware of image capture hardware 12.

[0040] According to some implementations consistent with this disclosure, the control logic can operate an actuator subassembly of the drone 4 to activate or press a button on the image capture hardware 12, if the image capture hardware 12 is a separate camera physically coupled to the drone 4. According to other implementations consistent with this disclosure, the control logic can operate logic in the image capture hardware 12 to activate image capture capabilities, if the image capture hardware 12 is integrated into the drone 4. The image capture hardware 12 can then provide the captured digital image(s) to processing circuitry of the drone 4 and / or to processing circuitry of the computing system 8 over various types of communication channels suitable for transferring digital image data using wireless or wired means.

[0041] As used herein, a processing circuit may include one or more of a central processing unit (CPU), a graphics processing unit (GPU), a single-core or multi-core processor, a controller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a fixed function circuit, a programmable circuit, any combination of fixed function and programmable circuitry, a discrete logic circuit, or an integrated logic circuit. The processing circuit of the drone 4 or computing system 8 may analyze the image(s) received from the image capture hardware 12 according to a trained ML model and detect misapplication of the tape 14 (or a portion thereof) applied to the substrate 16 based on the analysis. In various embodiments of the present disclosure, the processing circuit may execute a trained classification model, a trained detection model, or a trained segmentation model.

[0042] In some embodiments, the processing circuitry of the drone 4 or computing system 8 can leverage cloud computing capabilities to execute the trained model. In various embodiments, the trained model can be a trained deep learning model, such as a deep neural network. One example of a trained deep neural network that the processing circuitry can execute to analyze images of the tape 14 in accordance with the present disclosure is a trained convolutional neural network (CNN), which applies computer vision-directed machine learning techniques to detect misapplication of the tape 14 applied to the substrate 16. Aspects of the CNN are described in International Patent Applications having Publication Nos. WO2021 / 033061A1 and WO2020 / 003150A2, the entire disclosures of each of which are incorporated herein by reference. One example of a trained CNN that the processing circuitry of the drone 4 or computing system 8 can execute to perform defect detection aspects of the present disclosure is Mask R-CNN.

[0043] 3A and 3B are conceptual diagrams showing further details of misapplication of tape 14 to substrate 16 that embodiments of system 10 can detect using techniques of the present disclosure. FIG. 3A shows an example of a tape application defect that a processing circuit of drone 4 or computing system 8 can detect using a trained model of the present disclosure. Some of the defects shown in FIG. 3A correspond to what is referred to herein as a "fishmouth wrinkle." As used herein, a fishmouth wrinkle refers to a misapplication of tape involving a non-adhered portion of the edge of tape 14 from substrate 16 with adjacent non-adhered or non-coplanar portions of the inner portion of tape 14 with substrate 16. The opening caused by the non-adhered portion of the edge of tape 14 applied to substrate 16 and the adjacent protruding wrinkle creates an opening through which fluids (air, water, etc.) can enter and impair the function of tape 14 applied to substrate 16.

[0044] Another type of defect that the processing circuitry of the drone 4 or computing system 8 can detect by executing a trained model includes "tenting," which refers to non-adhered portions and protrusions of the inward protrusions of the tape 14 against the substrate 16 (e.g., without the edge penetration points of the fishmouth wrinkles shown in FIG. 3A). Tenting of the tape 14 can be caused by an incorrect application procedure or by applying the tape 14 over an "underdriven fastener." An underdriven fastener refers to a nail, screw, bolt, or other type of fastener that is partially driven into the substrate 16, but with the head of the fastener protruding beyond the substrate 16 to an extent that causes the tape 14 to tent when applied over the head of the fastener.

[0045] FIG. 3B illustrates another example of a tape application defect that the processing circuitry of the drone 4 or computing system 8 can detect using the trained model of the present disclosure. The defect illustrated in FIG. 3A is referred to herein as a "missing tape segment." The missing tape segment 18 indicates a misapplication of the tape 14 that leaves a portion of the substrate 16 exposed when it should not be exposed, such as due to indoor conditioning issues. For example, the missing tape segment 18 may expose a seam or substrate patch of the substrate that should not be exposed to the elements. The processing circuitry of the drone 4 or computing system 8 can also execute the trained model of the present disclosure to detect discontinuities in the tape 14 that are different from the missing tape segment 18. As an example, the processing circuitry of the drone 4 or computing system 8 can execute the trained model of the present disclosure to detect tears that do not span the entire width of the tape 14, or scratches that do not expose the substrate 16 underneath the tape 14, but instead impair or reduce the effectiveness of the tape 14. The processing circuitry of the drone 4 or computing system 8 may also execute the trained models of the present disclosure to analyze images received from the image capture device 12 and detect other types of misapplication of the tape 14 applied to the substrate 16, such as insufficient adhesion (beyond the examples discussed above), insufficient tension (e.g., as the tape 14 is not wound down with enough force as it is applied to the substrate 16, which can cause "slack" for the tape 14), etc. In some examples, the processing of the drone 4 or computing system 8 (whether local or leveraging cloud computing resources) may analyze the images of the tape 14 to determine brand information, regulatory compliance information, etc., regarding the tape 14, regardless of whether that information is represented in a manner visible to the human eye without image processing.

[0046] 4A-4D are diagrams illustrating various deep learning generated image labels that the trained model of the present disclosure can generate. Each of FIG. 4A-4D illustrates a model output that the processing circuitry of the drone 4 or computing device 8 can generate by executing the trained model of the present disclosure at various levels of computational complexity. FIG. 4A-4D are arranged in ascending order of computational complexity for generating the image labels of the present disclosure.

[0047] In the example of FIG. 4A, the trained classification model of the present disclosure performs a whole image classification on an image of the tape 14 applied to the substrate 16. In this example, the processing circuitry of the drone 4 or computing system 8 returns a “fail” classification in the form of an image label 19. The “fail” result of the image label 19 is a result or model output generated by the trained classification model based on detecting at least one defect anywhere in the image received from the image capture hardware 12. In the example of FIG. 4B, the trained detection model of the present disclosure performs a sub-image classification on the image of the tape 14 applied to the substrate 16. In this example, the trained detection model of the present disclosure divides the image into multiple sub-images 20 and classifies each respective sub-image 20 with a “pass” or “fail” label as was done for the entire image as a whole in the example of FIG. 4A by the image label 19.

[0048] In the example of Figure 4C, the trained detection model of the present disclosure performs object detection on an image of tape 14 applied to substrate 16. According to the object detection based technique shown in Figure 4C, the trained detection model of the present disclosure returns a rectangular bounding box (bounding box 22) around an area of ​​the image that shows defects in tape 14 applied to substrate 16, for which the trained detection model of the present disclosure returns a respective "fail" result as a model output. As shown in the non-limiting example of Figure 4C, the trained detection model can perform object detection and return multiple (potentially overlapping, as in the case of Figure 4C) bounding boxes 22.

[0049] In the example of Figure 4D, a trained segmentation model of the present disclosure performs image segmentation on an image of tape 14 applied to substrate 16. In accordance with the image segmentation-based technique illustrated in Figure 4D, a trained segmentation model of the present disclosure returns a pixel-by-pixel labeling of defects related to tape 14 applied to substrate 16 as represented in an image obtained from image capture hardware 12. Figure 4D illustrates defect segments 24 that a trained segmentation model of the present disclosure identifies based on a pixel-by-pixel analysis of an image received from image capture hardware 12 showing tape 14 applied to substrate 16.

[0050] 3A-4D represent images that, in various embodiments, are represented in various color spaces, such as red-green-blue (RGB) color space, grayscale color space, monochrome space, or various other chromaticity spaces partially or fully distinguishable to the human eye. In these embodiments, image capture device 12 represents digital camera hardware configured to generate digital image data in any of these color spaces. In other embodiments, image capture hardware 12 may represent a so-called "polarized camera." A polarized camera may generate image data in various formats by performing calculations on data output by a polarization sensor of the polarized camera.

[0051] FIG. 5 is a conceptual diagram illustrating a polarized image that the system 10 can analyze to detect defects associated with the tape 14 applied to the substrate 16, according to an embodiment of the present disclosure. In an embodiment in which the image capture hardware 12 represents a polarized camera, the processing circuitry of the drone 4 or computing system 8 can analyze various data output by the image capture hardware 12 to detect defects associated with the tape 14 applied to the substrate 16. As an example, the processing circuitry of the drone 4 or computing system 8 can execute a trained classification model of the present disclosure to analyze two values: unpolarized light in the polarized image, and the degree of linear polarization (DoLP) shown in the polarized image. FIG. 5 illustrates a pre-processed or processed polarized image with DoLP data calculated for each pixel in each color channel that the trained model of the present disclosure can utilize to detect with high confidence the application defects based on the illustrated wrinkles in the tape 14.

[0052] In some non-limiting examples where the image capture hardware represents a polarization camera, the image sensor hardware of image capture hardware 12 includes four polarization filters oriented at 0 degrees, 45 degrees, 90 degrees, and 135 degrees, respectively. Using the notation where the four images obtained through the four polarization filters are represented by I0, I45, I90, and I135, the polarization camera of image capture hardware 12 can calculate the Stokes vectors (S0, S1, S2, and S3) for each color channel according to the following calculation:

number

[0053] The Stokes vectors calculated according to the above equation (1) respectively represent the intensity image of unpolarized light (S0), linearly or horizontally polarized light (S1), the intensity image of light polarized at 45 degrees or 135 degrees (S2), and circularly polarized light (S3). The polarization camera can calculate the DoLP using the above mentioned Stokes vectors according to the following equation (2):

number

[0054] The processing circuitry of the drone 4 or computing system 8 can execute the trained models of the present disclosure to use linear polarization to detect wrinkle-based defects, such as wrinkles 26 and / or fishmouth wrinkles 28, on the tape 14 applied to the substrate 16. Because the surface of the tape 14 reflects light differently depending on the angle of the reflected incident light, the presence and potentially magnitude (with respect to the angle outward from the substrate 16) of wrinkles 26 and / or fishmouth wrinkles 28 alters the polarization-based measurements described above.

[0055] Once implemented, the trained model of the present disclosure can use DoLP calculated according to Equation (2) to measure the consistency of light reflection, regardless of the directionality of the light reflection. In various experiments, the tape 14 had a black and glossy appearance. Regardless of the color of the tape 14, the trained model of the present disclosure can leverage the gloss properties of the tape 14 to use DoLP to detect wrinkle shading and other effects and detect defects in the application of the tape 14 to the substrate 16. A darker color of the tape 14 (such as the black color used in the experiments above) can further enhance the ability of the trained model of the present disclosure to use DoLP to detect wrinkles 26 and / or fishmouth wrinkles 28 in the tape 14 applied to the substrate 16.

[0056] 6A-6D are diagrams illustrating various deep learning generated image labels that a trained model of the present disclosure can generate using the polarized image shown in FIG. 5. Each of FIG. 6A-6D illustrates image labels that a processing circuit of the drone 4 or computing device 8 can generate by running a trained model of the present disclosure on the polarized image of FIG. 5 at various levels of computational complexity. FIG. 6A-6D sequentially illustrate an ascending order of computational complexity for generating image labels using the polarized image of FIG. 5.

[0057] In the example of Figure 6A, a trained classification model of the present disclosure performs a full image classification on a polarized image of tape 14 applied to substrate 16. In this example, the processing circuitry of drone 4 or computing system 8 returns a "fail" result based on a full image analysis of the polarized image, the conditions met upon detection of the first detected wrinkle 26 or fishmouth wrinkle 28, or both in the case of substantially simultaneous detection. The "fail" result of the model output is indicated by image label 29 in Figure 6A.

[0058] In the example of Figure 6B, a trained detection model of the present disclosure performs sub-image classification on a polarized image of tape 14 applied to a substrate 16. In this example, the trained detection model of the present disclosure divides the polarized image into multiple sub-images 30 and classifies each respective sub-image 30 as a "pass" or "fail" result, as was done for the entire polarized image as a whole in the example of Figure 6A.

[0059] In the example of Figure 6C, a trained detection model of the present disclosure performs object detection on a polarized image of tape 14 applied to substrate 16. According to the object detection based technique illustrated in Figure 6C, a trained detection model of the present disclosure returns a rectangular bounding box (bounding box 32) around an area of ​​the polarized image that shows defects in tape 14 applied to substrate 16. As shown in the non-limiting example of Figure 6C, the trained detection model can perform object detection and return multiple (and potentially overlapping) bounding boxes 32.

[0060] In the example of Figure 6D, a trained segmentation model of the present disclosure performs image segmentation on a polarized image of tape 14 applied to substrate 16. According to the image segmentation-based technique illustrated in Figure 6D, a trained segmentation model of the present disclosure returns pixel-by-pixel labeling of wrinkles 26 and fishmouth wrinkles 28 for tape 14 applied to substrate 16 as depicted in an image acquired from a polarized camera of image capture hardware 12. Figure 6D illustrates defect segments 24 that a trained segmentation model of the present disclosure identifies based on a pixel-by-pixel analysis of light polarization shown in the polarized image of Figure 5 showing tape 14 applied to substrate 16.

[0061] 7 is a graph 36 illustrating an aspect of polarized image analysis that a trained model of the present disclosure may perform to detect one or more defects on a tape 14 applied to a substrate 16. Plots 38, 40, and 42 show the change in true positive rate (for image classification) as a function of the corresponding false positive rate under different polarized image analysis scenarios. Overall, graph 36 represents a receiver operator characteristic (ROC) curve generated from a test data set classified by a trained model of the present disclosure for DoLP images and unpolarized (S0) images.

[0062] As shown in graph 36, the area under the curve (AUC) is largest for DoLP plot line 38 (corresponding to polarized images) when compared to the AUC for S0 plot line 40 (corresponding to unpolarized images) and random plot line 42 (provided as baseline ground truth). The AUC shown in FIG. 7 provides an overall measure of how distinguishable the corresponding classes of images are in the dataset provided to the trained model of the present disclosure. The AUC for DoLP plot line 38 is largest among the plot lines shown in graph 38, indicating that the trained model of the present disclosure can distinguish wrinkles 26 and / or fishmouth wrinkles 28 from the DoLP images more easily than using other images.

[0063] In this manner, the techniques of the present disclosure improve the accuracy of defect detection on tape 14 applied to substrate 16 by using trained models (e.g., one or more of classification, detection, or segmentation models) of the present disclosure. Whether image capture hardware 12 provides images in RGB color space, grayscale color space, or as polarized light images (or DoLP images), the trained models of the present disclosure detect various types of defects on tape 14 applied to substrate 16 while improving data accuracy (e.g., by mitigating human error resulting from different visual acuity or perceptual capabilities). Although primarily described as being coupled to or incorporated into drone 4 as a non-limiting example use case, it will be understood that the trained model-based image analysis techniques of the present disclosure also provide these data accuracy improvements in non-drone-based implementations as well.

[0064] For example, the trained model of the present disclosure can use images captured by image capture hardware 12 in embodiments where image capture hardware 12 is incorporated into a mobile computing device, such as a smartphone, tablet computer, wearable computing device, etc. As another embodiment, the trained model of the present disclosure can use images captured by image capture device 12 where image capture device 12 is a dedicated digital camera or a dedicated polarized camera. In any of the non-drone-based embodiments listed above, the trained model of the present disclosure can use images of tape 14 applied to substrate 16 based on manual capture of the image, such as by user input provided via an actuator button on a digital camera or touch input provided on a touch screen of a mobile computing device.

[0065] In accordance with the drone-hosted implementations described above, the system of the present disclosure also improves safety and the ability to capture and analyze images from difficult to access areas of the substrate 16. For example, by using the drone 4 to transport image capture hardware 12 to potentially dangerous locations and capture images at these locations, the system 10 reduces or potentially eliminates the need to put human workers at risk by requiring them to access these locations for manual image capture. The drone 4 can also provide piloting capabilities that would not otherwise be available to equipment used by workers to survey the substrate 16, thereby improving accessibility and tape inspection capabilities for these areas of the substrate 16.

[0066] 8 is a conceptual diagram illustrating an embodiment of drone-hosted substrate inspection of the present disclosure. Based on navigation instructions received from drone controller 6 or computing system 8, control logic of drone 4 can cause a motion guide to navigate drone 4 to an area proximate to a particular area of ​​substrate 16. Upon detecting that drone 4 is positioned and oriented sufficiently to allow image capture by image capture hardware 12, the control logic can activate image capture hardware 12 to capture one or more images of a portion of substrate 16 that is within the field of view of the lens hardware of image capture hardware 12.

[0067] The processing circuitry of the drone 4 or computing system 8 can analyze the image(s) received from the image capture hardware 12 by executing any of the trained models described above (e.g., one or more of a classification model, a detection model, or a segmentation model) to detect defects in the substrate 16. As described above, in various embodiments, the trained model may be a trained deep neural network, such as a trained CNN. In these and other embodiments, the trained models of the present disclosure can apply computer vision-directed machine learning techniques to detect defects in the substrate 16.

[0068] 9A-9C are conceptual diagrams illustrating examples of underdrive fasteners in a substrate 16 that a trained model of the present disclosure may detect as a substrate defect. As used herein, the term "underdrive fastener" may refer to any nail, screw, bolt, tack, or other through-hole fastener that is not driven deep enough or sufficiently uniformly into the substrate 16 to place the fastener head 44 substantially flush with the surface of the substrate 16. Underdrive fasteners compromise the structural integrity of the building envelope or other structure represented by the substrate 16. If tape 14 is applied over the portion of the substrate 16 that surrounds and includes the underdrive fastener shown in FIG. 9A, the protrusion of the fastener head 44 may result in misapplication of the tape 14 against the substrate 16 based on tenting.

[0069] An embodiment of the system 10 can capture an image of the substrate 16 shown in FIG. 9A based on positioning the image capture hardware 12 sufficiently close to the substrate 16 (e.g., by using the drone 4 as shown in FIG. 8 or via manual positioning as described above with respect to other embodiments) and activating the image capture hardware 12 to capture an image. The drone 4 or processing circuitry of the computing system 8 can execute a trained model of the present disclosure using the image of FIG. 9A to detect underdriven fasteners based on image data representative of the position and / or orientation of the fastener head 44. In this manner, the trained model of the present disclosure, during its execution phase, can provide a model output indicative of the underdriven condition of the fastener of FIG. 9A, thereby enabling timely repair of the underdriven fastener.

[0070] For example, an underdriven fastener that the trained model of the present disclosure detects based on the location and / or orientation of the fastener head 44 can be repaired based on the model output provided by the trained model of the present disclosure before further construction-related tasks are performed on the substrate 16. In this example, the trained model of the present disclosure reduces or potentially eliminates the need for additional disassembly or disassembly simply to access the underdriven fastener prior to repair. Instead, by detecting the underdriven fastener based on analyzing image data representative of the fastener head 44 during skin layer inspection, the trained model of the present disclosure enables repair of the underdriven fastener in a timely and efficient manner.

[0071] The image shown in FIG. 9B illustrates a defect in the effectiveness of tape 14 when applied over an underdrive fastener defect in substrate 16 indicated by the protrusion of fastener head 44 in FIG. 9A. In examples where a trained model of the present disclosure is performed using the image of FIG. 9B, the model output enables various types of repairs, such as a sequence of removal of tape 14, repair of the underdrive fastener evidenced by the location and / or orientation of fastener head 44, and reapplication of a new segment of tape 14 to the repaired substrate 16. In some examples, such as examples where the trained model is also trained to detect defects in the application of tape 14 to substrate 16 (as described above with respect to FIGS. 1-6), the trained model can also communicate model output indicative of a misapplication of tape 14 (in this particular example, a tear) at the location of fastener head 44.

[0072] The image shown in FIG. 9C illustrates a defect in the effectiveness of tape 14 when applied over an underdrive fastener defect in substrate 16, indicated by tenting 47 of tape 14 applied to substrate 16. Tenting 47 occurs because tape 14 is applied over an underdrive fastener that is improperly embedded in substrate 16 and the underdrive fastener does not have tension to break or penetrate tape 14. In an example where a trained model of the present disclosure is run using the image of FIG. 9C, the model output allows for various types of repairs, such as a sequence of removal of tape 14, repair of tenting 47, such as by removing the underdrive fastener or driving the underdrive fastener flush with substrate 16, and reapplication of a new segment of tape 14 to the repaired substrate 16 such that the new segment of tape 14 is flush with substrate 16.

[0073] 10A and 10B are conceptual diagrams illustrating examples of board separations in a substrate 16 that a trained model of the present disclosure can detect as a substrate defect. As used herein, the term "separation" refers to a non-flush joint between two adjacent boards of a substrate 16, such as non-flush joint 45. A non-flush joint 45 can represent a gap between boards that are not bonded together tightly enough, or can represent a slope difference between adjacent boards that are located at different depths, or can represent a combination of these defects. Board separations resulting from conditions such as non-flush joint 45 compromise the structural integrity of a building envelope or other structure represented by the substrate 16.

[0074] An embodiment of the system 10 can capture an image of the substrate 16 shown in FIG. 10A based on positioning the image capture hardware 12 sufficiently close to the substrate 16 (e.g., by using a drone 4 as shown in FIG. 8 or via manual positioning as described above with respect to other embodiments) and activating the image capture hardware 12 to capture an image. The drone 4 or processing circuitry of the computing system 8 can execute one or more of the trained models of the present disclosure using the image of FIG. 10A to detect the presence of a non-coplanar joint 45. In this manner, the trained models of the present disclosure can provide model outputs during their execution phase that indicate the presence of a non-coplanar joint 45, enabling timely repair of the resulting board separation. In various embodiments, the model outputs of the trained models of the present disclosure can enable various types of repair, such as manual repair, automated repair (e.g., using a drone or other equipment), or any other suitable repair scheme or mechanism.

[0075] For example, a board separation caused by a non-coplanar joint 45 can be repaired based on the model output provided by the trained model(s) of the present disclosure before further construction-related tasks are performed on the substrate 16. In this example, the trained model of the present disclosure reduces or potentially eliminates the need for additional disassembly or disassembly simply to access the non-coplanar joint 45 prior to repair. Instead, by detecting the board separation caused by the non-coplanar joint 45 during skin layer inspection, the trained model of the present disclosure enables repair of the board separation in a timely and efficient manner.

[0076] The image shown in FIG. 10B illustrates defects in the effectiveness of tape 14 when applied over a board separation defect in substrate 16 illustrated by non-coplanar joint 45 in FIG. 10A. In examples where a trained model of the present disclosure is performed using the image of FIG. 10B, the model output enables various types of repairs, such as a sequence of removal of tape 14, repair of the board separation resulting from non-coplanar joint 45, and reapplication of a new segment of tape 14 to the repaired substrate 16. In some examples, such as examples where the model is also trained to detect defects in the application of tape 14 to substrate 16 (as described above with respect to FIGS. 1-7), the trained model can also communicate model output indicative of non-adhered portions of tape 14 at non-coplanar joint 45.

[0077] FIG. 11 is a conceptual diagram illustrating an example of a defect in substrate 16 caused by an overdrive fastener and detected using the trained model of the present disclosure. As used herein, the term "overdrive fastener" refers to a nail, screw, tack, bolt, or other fastener that is driven to an excessive depth such that the head or other type of proximal end of the fastener penetrates substrate 16 and is currently located below the substrate. Overdriving a fastener causes a hole 50 to form in the surface of substrate 16 (and to a depth below the surface). Hole 50 represents a defect that may compromise the structural integrity of substrate 16 and the integrity of substrate 16 with respect to insulating the interior of building 2 from weather conditions such as temperature, water, and other elements. Hole 50 may cause heat transfer, water ingress, or other degradation of functionality to substrate 16. Although hole 50 is described herein as being caused by an overdrive fastener as an example, hole 50 may also be caused by other factors such as wind-blown debris, dislodged fasteners, etc.

[0078] The processing circuitry of the drone 4 or computing system 8 can execute the trained model of the present disclosure using the image of FIG. 11 to detect the presence of the hole 50. In this manner, the trained model(s) of the present disclosure can provide a model output indicative of the presence of the hole 50 during its respective execution stage(s), allowing for timely repair of the resulting board separation. The trained model(s) of the present disclosure can also provide a documentation trail for construction site managers, inspectors, contractors, and the like, thereby assisting construction management in providing information relevant to insurance and potentially clarifying disputed items in future disputes. In this example, the trained model of the present disclosure reduces or potentially eliminates the need for additional dismantling or disassembly to simply access the hole 50 prior to repair. Instead, by detecting the structural defect of the substrate 16 represented by the hole 50 during the outer skin layer inspection, the trained model of the present disclosure allows for the repair of the hole 50 in a timely and efficient manner.

[0079] 12A and 12B are conceptual diagrams illustrating examples of impact-related damage in a substrate 16 that a trained model of the present disclosure can detect as a substrate defect. As used herein, the term "impact-related damage" refers to any type of damage to a substrate 16 that may result from striking (inward pressure) or gouging (outward pressure). Although a substrate 16 may exhibit impact-related damage resulting from a number of different causes, the examples of FIG. 12A and 12B are described herein with respect to damage caused by hammer claw end gouging and hammer strikes, as non-limiting examples. The impact-related damage shown in FIG. 12A and 12B compromises the structural integrity of a building envelope or other structure represented by the substrate 16.

[0080] An embodiment of the system 10 can capture an image of the substrate 16 shown in FIG. 12A based on positioning the image capture hardware 12 sufficiently close to the substrate 16 (e.g., by using a drone 4 as shown in FIG. 8 or via manual positioning as described above with respect to other embodiments) and activating the image capture hardware 12 to capture an image. The drone 4 or processing circuitry of the computing system 8 can execute the trained model of the present disclosure using the image of FIG. 12A to detect the presence of a surface tear 46. The trained model of the present disclosure, in its execution phase, can detect tears of various widths and severity. As shown in FIG. 12A, the trained model of the present disclosure detects two relatively large tears, as well as multiple smaller tears or "dents" in the substrate 16. In this manner, the trained model of the present disclosure, in its execution phase, can provide a model output indicative of the presence of the surface tear 46, enabling repair of the surface tear 46 in a timely manner.

[0081] For example, board separation caused by non-coplanar joints 45 can be repaired based on model output provided by the trained model of the present disclosure before further construction-related tasks are performed on the substrate 16. In this example, the trained model of the present disclosure reduces or potentially eliminates the need for additional disassembly or disassembly simply to access the non-coplanar joints 45 prior to repair. Instead, by detecting board separation caused by non-coplanar joints 45 during skin layer inspection, the trained model of the present disclosure enables repair of the board separation in a timely and efficient manner.

[0082] The image shown in Figure 12B illustrates a surface indentation 48 on the substrate 16. The surface indentation 48 may be caused by excessive force and / or an improper angle applied when striking the surface of the substrate 16 with a hammer, or by other factors. In an example where the trained models of the present disclosure are run using the images of Figure 12B, the respective model output identifies a surface indentation 48, which illustrates an instance where the hammer strikes exposed material (e.g., wood) located beneath a weathering coating applied to the substrate 16.

[0083] 13A and 13B show examples in which a drone 4 is equipped and configured to mark potential objects of interest (OoPSI) on a substrate 16 or tape 14 according to aspects of the disclosure. In some examples, aspects of the system 10 can navigate the drone 4 to an area near an OoPSI identified using the trained model described above with respect to FIGS. 1-12B or otherwise identified. In the example of FIG. 13A, the drone 4 comprises a top mount 52. The top mount 52 can represent any hardware or combination of hardware components that, when physically coupled to the top surface of the drone 4 (directed in flight), allows for further coupling of the drone 4 to additional attachments and components. In other embodiments, the drone 4 can comprise a bottom mount that allows for coupling of additional attachments and / or components via a surface of the drone 4 that faces the ground in flight.

[0084] The drone 4 includes a shock absorbing subassembly 54. In the embodiment of FIG. 13A, the upper mount 52 couples the drone 4 to the shock absorbing subassembly 54. In some embodiments, the shock absorbing subassembly 54 represents a compression spring set that may include a single compression spring or multiple compression springs. In other embodiments, the shock absorbing subassembly 54 may represent other types of shock absorbing technologies, such as hydraulic devices, compression bladders, struts, magnetic fluids, and the like. In any case, the shock absorbing subassembly 54 is configured to absorb and / or attenuate the impact impulse by converting the impact-related shock into another form of energy that can be dissipated, such as thermal energy.

[0085] The drone 4 also includes a marking device 56. The marking device 56 can represent various types of equipment configured to mark an area of ​​the substrate 16 or an area of ​​the tape 14 applied to the substrate 16. In one embodiment, the marking device 54 represents an ink delivery system, such as a pen, felt tip pen, marker, Bingo Dauber, etc., configured to deliver ink upon contact between a distal tip of the marking device 56 and a receptacle, such as the substrate 16 or the tape 14 applied to the substrate 16. In another embodiment, the marking device 56 is configured to deliver a self-tack paper strip onto a receptacle (e.g., the substrate 16 or the tape 14 applied to the substrate 16) that the distal tip of the marking device 56 contacts. In other embodiments, the marking device 56 is configured to mark a receptacle (such as the substrate 16 or the tape 14 applied to the substrate 16) in other ways.

[0086] FIG. 13B shows further details of a particular embodiment of the drone 4 configured in the example of FIG. 143. FIG. 13B is a side view of various components coupled to the drone 4 via the upper mount 52. FIG. 13B shows a compression range 58 of the shock absorbing subassembly 54. The compression range 58 represents the length over which the shock absorbing subassembly 54 allows for a temporary reduction in the overall length of the combination of components coupled to the drone 4 via the upper mount 52.

[0087] It will be appreciated that the compression range 58 does not represent the length that the impact absorbing subassembly 54 compresses each time the marking device 56 impacts an object, such as the substrate 16. Rather, the compression range 58 represents the maximum compression imparted by the impact absorbing subassembly 54 when the distal tip of the marking device 56 contacts a rigid or semi-rigid body (e.g., the substrate 16 or the tape 14 applied to the substrate 16). In the example orientation shown in FIG. 13B, the right end of the marking device 56 includes the distal tip that contacts the substrate 16 as part of the OoPSI marking function described herein.

[0088] Depending on the force of the impact, the shock absorbing subassembly 54 can be compressed either to the full extent of the compression range 58 or below the compression range 58. In the configuration shown in FIG. 13B, the shock absorbing subassembly 54 is disposed between the marking mount 60 and the rear stop 64. The marking mount 60 represents a component configured to receive the marking device 56. In some embodiments, the marking mount 60 has an expandable or configurable diameter and / or shape, thereby enabling the marking mount 60 to receive marking devices or other peripherals of various shapes, sizes, form factors, etc.

[0089] In this manner, the marking mount 60 allows for the use of various types of marking peripherals with the systems and techniques of the present disclosure. The rear stop 64 represents a rigid component having a fixed position. The rear stop 64 allows the drone 4 to provide a reaction force against a collision between the distal tip marking device 56 and the substrate 16 or tape 14 while accommodating the compression provided by the shock absorbing subassembly 54 up to a maximum length represented by the overall length of the compression range 58.

[0090] 13B, the drone 4 also includes a motion guide 66. In various embodiments, the motion guide 66 is a linear motion guide that provides a sliding framework for the reciprocating motion of the marking mount 60 (which holds the marking device 56) in response to a distal tip of the marking device 56 impacting the substrate 16. The motion guide 66 is coupled to the drone 4 via the top mount 52 and holds the shock absorbing subassembly 54 in place between the motion guide 66 and the marking mount 60 using one or more fasteners (e.g., in a slotted or another type of channel).

[0091] The control circuitry of the drone 4 is configured to navigate the drone 4 to an area associated with the identified OoPSI (e.g., at, including, or proximate to the identified OoPSI). The control circuitry can accomplish these movements of the drone 4 using a local position tracker and other hardware of the drone 4. For example, the control circuitry can navigate the drone 4 to an area associated with the identified OoPSI based on instructions received from the control logic of the drone 4. The control logic of the drone 4 can then navigate the drone 4 to the area associated with the OoPSI based on navigation instructions that the control logic receives from the drone 4 or processing circuitry of the computing system 8.

[0092] In some embodiments, the drone 4, as configured in the embodiment of Figures 13A and 13B, can navigate to and mark an OoPSI associated with a defect in the substrate 16, such as the examples shown and described with respect to Figures 8, 9A, 10A, and 11-12B. Examples of substrate defect OoPSI that the drone 4 can navigate to and mark according to the embodiment of Figures 13A and 13B include surface tears, underdriven fasteners, overdriven fasteners, surface gouges, excess sealant, board separations, gaps, etc.

[0093] In some embodiments, the drone 4, as configured in the embodiment of Figures 13A and 13B, can navigate to and mark OoPSIs associated with tape misapplication(s) for tape 14 applied to substrate 16, such as the examples shown and described with respect to Figures 2-7, 9B, and 10B. Examples of tape misapplication related OoPSIs that the drone 4 can navigate to and mark according to the embodiment of Figures 13A and 13B include fishmouth wrinkles, tenting of tape 14 applied to substrate 16, missing segment(s), various types of insufficient adhesion, insufficient tension, etc.

[0094] 14A-14C are conceptual diagrams illustrating examples in which a drone 4 is equipped and configured to repair an OoPSI on a substrate 16 or a tape 14 applied to a substrate 16, according to embodiments of the present disclosure. In some embodiments, embodiments of the system 10 can navigate the drone 4 to an area near an OoPSI identified using the trained model described above with respect to FIGS. 1-12B or otherwise identified. In the embodiment of FIG. 14A, the drone 4 comprises an upper mount 52 and a lower mount 68. The lower mount 68 can represent any hardware or combination of hardware components that, when physically coupled to an underside or ground-facing surface of the drone 4 (in its in-flight orientation), allows for further coupling of the drone 4 to additional attachments and components.

[0095] Via the lower mount 68, the drone 4 comprises a dispenser subassembly 72. The dispenser subassembly 72 includes a housing 75 that receives a syringe 76. As shown, the housing 75 is configured to receive the syringe 76 in a position and orientation such that an applicator of the syringe 76 is disposed distally from the housing 75. Thus, the dispenser subassembly 72 is configured to accommodate the syringe 76 in a position and orientation that allows any contents of the syringe 76 to be expelled distally from the body of the drone 4.

[0096] 13B , the control circuitry of the drone 4 is configured to navigate the drone 4 to an area associated with the identified OoPSI (e.g., at, including, or proximate to the identified OoPSI) based on instructions that the control logic of the drone 4 generates based on navigation instructions received from the processing circuitry of the drone 4 or the computing system 8. The control logic of the drone 4 can also receive extrusion instructions from the processing circuitry of the drone 4 or the computing system 8.

[0097] 14B shows a top view of the dispenser subassembly 72. Based on the push command received from the processing circuit, the control logic can actuate the actuator motor 77. For example, based on the received push command, the control logic of the drone 4 can cause the actuator motor 77 to move the actuator arm 80 to an extended phase of reciprocation. The extended phase of reciprocation represents a phase in which the actuator arm 80 moves on a linear path distal to the body of the drone 4. The specified distance 82 refers to the distance the actuator can move within the dispenser subassembly 72, which can correlate to material wastage in this form factor.

[0098] By moving the actuator arm 80 to the extended phase of reciprocation, the actuator motor 77 causes the actuator arm 80 to extrude a portion of the contents of the syringe 76. Based on the drone 4 being located in an area associated with the identified OoPSI, the actuator motor 77 causes the actuator arm 80 to extrude the contents of the syringe 76 to the area associated with the OoPSI. In some embodiments, based on the navigation command and / or the extrusion command, the control logic of the drone 4 is configured to move the drone 4 parallel or substantially parallel to the surface of the substrate 16 while the actuator arm 80 is in the extended phase of reciprocation to extrude the contents of the syringe 76.

[0099] As used herein, movement of drone 4 substantially parallel to the surface of substrate 16 refers to movement in any pattern substantially parallel to the XY plane of substrate 16. By moving drone 4 substantially parallel to the XY plane of substrate 16 while actuator arm 80 is in the extended stage, control logic of drone 4 processes navigation and extrusion commands to extrude the contents of syringe 76 onto some, most, or all of the identified OoPSI.

[0100] That is, the navigation command can correspond to a movement pattern that, upon completion, covers some, most, or all of the identified OoPSI. In some embodiments, based on the extrusion increment associated with the extrusion command, the control logic of the drone 4 can cause the actuator motor 77 to move the actuator arm 80 to a retraction phase of reciprocation to cease extrusion of the contents of the syringe 76. That is, the extrusion increment can define an amount of the contents of the syringe 76 that is extruded to repair the OoPSI, assuming movement of the drone 4 to cover a sufficient area of ​​the OoPSI while the contents of the syringe 76 are being extruded.

[0101] The actuator coupler 74 physically couples the distal end of the actuator arm 80 (relative to the body of the drone 4) to the proximal end of the syringe 76 (relative to the body of the drone 4) and causes the proximal end of the syringe 76 to track both the extension and retraction phases of the reciprocating motion of the actuator arm 80.

[0102] FIG. 14C shows further details of the slotted channel 70 shown in FIG. 14A. As shown in FIG. 14A, the slotted channel 70 is configured to couple the dispenser subassembly 72 to the body of the drone 4. The slotted channel 70 provides a self-weighted uncontrolled degree of freedom (DOF) 88 of radial movement of the dispenser subassembly 72 relative to a reference point of a fixed point to the body of the drone 4. By providing an uncontrolled DOF 88 of movement of the dispenser subassembly 72, the slotted channel 70 provides an error buffer (e.g., headwind gusts, rotor washing, etc.) for the radial movement of the dispenser subassembly 72.

[0103] The uncontrolled DOF 88 provided by the slotted channel 70 reduces the need for additional motors and on-board component infrastructure that would be required for a controlled DOF implementation, thus adding weight to a potentially weight-sensitive system. However, it will be understood that controlled DOF implementations and / or rigidly fixed implementations are also consistent with the adhesive supply drone implementations of the present disclosure. FIG. 14C shows the pivot hub 84 and radial fasteners 86A and 86B. The radial fasteners 86A and 86B are positioned equidistant from the pivot hub 84 and provide the arc contained within the uncontrolled DOF 88.

[0104] For ease of illustration only the arc represented by the circumferential movement between radial fasteners 86A and 86B is shown in FIG. 14C, however it will be appreciated that slotted channel 70 can include a varying number of radial fasteners to provide uncontrolled DOF 88. In various embodiments, syringe 76 can be loaded with various types of adhesive contents such as caulking material, general purpose silicone adhesive, nitrocellulose adhesive, paste sealant, epoxy acrylic, or other adhesive suitable for dispensing using dispenser subassembly 72. In some embodiments, drone 4 can be equipped with replaceable syringes, with syringe 76 representing the syringe currently in use, with other backup and / or used syringes also on board.

[0105] The embodiment of drone 4 shown in Figures 14A-14C can deliver the adhesive contents of syringe 76 to repair various types of OoPSI, including, but not limited to, defects in substrate 16 such as surface tears, underdriven fasteners, overdriven fasteners, surface gouges, gaps or other discontinuities between boards, collision-related damage, and / or misapplication of tape 14 such as fishmouth wrinkles, tears or scrapes, wrinkles, tenting, missing tape segments, insufficient adhesion, insufficient tension, etc.

[0106] FIG. 15 is a conceptual diagram illustrating another embodiment in which a drone 4 is equipped and configured to repair an OoPSI on a substrate 16 or a tape 14 applied to a substrate 16, according to an embodiment of the present disclosure. In some embodiments, an embodiment of the system 10 can navigate the drone 4 to an area near an OoPSI identified using the trained model described above with respect to FIGS. 1-12B or otherwise identified. In the embodiment of FIG. 15, the drone 4 includes a dispenser subassembly 90. The dispenser subassembly 90 includes a housing 94 that receives an aerosol delivery system 102. Although the dispenser subassembly 90 is shown in FIG. 15 as being attached to a top surface of the drone 4 as a non-limiting example, it will be understood that in other embodiments according to the present disclosure, the dispenser subassembly 90 can be coupled to the drone 4 in other ways.

[0107] Aerosol delivery system 102 can represent one or more types of canisters or storage devices configured to release compressed contents upon opening of a pressure valve, such as by depressing nozzle 104. As described with respect to FIG. 13B , control circuitry of drone 4 is configured to navigate drone 4 to an area associated with (e.g., at, including, or proximate to) the identified OoPSI based on instructions that control logic of drone 4 generates based on navigation instructions received from processing circuitry of drone 4 or computing system 8. Control logic of drone 4 can also receive delivery instructions from processing circuitry of drone 4 or computing system 8.

[0108] Based on the feed command received from the processing circuit, the control logic can actuate the motor 92. For example, based on the received feed command, the control logic of the drone 4 can cause the motor 92 to move the trigger 98 to a retraction phase of reciprocation. The retraction phase of reciprocation represents a phase in which the trigger 98 moves proximally toward the body of the drone 4. For example, when actuated in this manner by the control logic of the drone 4, the motor 92 can retract the link wire 96 toward the body of the drone 4, thereby retracting the trigger 98 coupled to the link wire 96.

[0109] Moving the trigger 98 in the retraction phase of the reciprocation causes the motor 92 to cause the trigger 98 to depress the nozzle 104, thereby expelling a portion of the contents of the aerosol delivery system 102. Based on the drone 4 being located in an area associated with the identified OoPSI, the motor 92 causes the trigger 98 to depress the nozzle 104, thereby dispensing the contents of the aerosol delivery system 102 to the area associated with the OoPSI. In some embodiments, based on the navigation command and / or the dispensing command, the control logic of the drone 4 is configured to move the drone 4 parallel or substantially parallel to the surface of the substrate 16 while the trigger 98 is in the retraction phase of the reciprocation, to depress and hold the nozzle 98, thereby dispensing the contents of the aerosol delivery system 102.

[0110] As used herein, movement of drone 4 substantially parallel to the surface of substrate 16 refers to movement in any pattern substantially parallel to the XY plane of substrate 16. By moving drone 4 substantially parallel to the XY plane of substrate 16 while trigger 98 is in the retraction phase of reciprocation, the control logic of drone 4 processes navigation and extrusion commands to deliver the contents of aerosol delivery system 102 onto some, most, or all of the identified OoPSI.

[0111] That is, the navigation command can correspond to a pattern of movement that, upon completion, covers some, most, or all of the identified OoPSI. In some embodiments, based on the feed increment associated with the extrusion command, the control logic of the drone 4 can cause the motor 92 to release at least a portion of the tension applied to the link wire 96 and move the trigger 98 to the extension phase of the reciprocating motion to cease the delivery of the contents of the aerosol delivery system 102. That is, the feed increment can define the amount of the contents of the aerosol delivery system 102 that is sprayed to repair the OoPSI, assuming movement of the drone 4 to cover a sufficient area of ​​the OoPSI while the contents of the aerosol delivery system 102 are being sprayed.

[0112] The contents of the aerosol delivery system 102 can include any aerosol propelled sealant or any other material suitable for being sprayed onto an identified OoPSI for sealing or molding purposes, such as rubber sealant, weather resistant spray paint, pressurized foam sealant, etc. The embodiment of the drone 4 shown in FIG. 15 can deliver the contents of the aerosol delivery system 102 to repair various types of OoPSI, including, but not limited to, defects in the substrate 16, such as surface tears, overdriven fasteners, surface gouges, gaps or other discontinuities between boards, collision related damage, and / or misapplication of tape 14, such as tears or scrapes, missing tape segments, poor adhesion, etc.

[0113] In some implementations consistent with FIGS. 14A-15, the drone 4 can include a light source, a light sensor, and a fiber optic link coupling the light source to the light sensor. In these implementations, the control logic of the drone 4 can activate the light source based on a feed / push command, and the motor 92 or actuator motor 77 (as the case may be) is configured to move the trigger 98 to the retraction phase or the actuator arm 80 to the extension phase of their respective reciprocating motion. In these examples, the control logic of the drone 4 uses these light-based techniques to depress the nozzle 104 or push the contents of the syringe 76 to deliver the aerosol delivery system 102 or the contents of the syringe 76 to the area associated with the OoPSI in response to the light sensor detecting activation of the light source via the fiber optic link.

[0114] In other implementations consistent with FIGS. 14A-15, drone 4 can include a microcontroller, a Bluetooth or other short-range, low-power, wireless transceiver, and a power source, such as a battery or battery pack. The microcontroller can continuously execute a script that, in the appropriate function call, initiates a connection with the wireless transceiver and transmits a signal corresponding to a feed increment or an extrusion increment. In these examples, the microcontroller-transceiver-based subsystem is separate and independent of the firmware of drone 4 and is therefore portable across and independent of different underlying UAV platforms, pending certain mechanical adjustments to fit the underlying UAV platform.

[0115] 16 is a flow chart illustrating an example process 110 of the present disclosure. The process 110 may begin with an inspection of the building 2 (106). For example, control logic of the drone 4 may navigate the drone 4 and activate the image capture hardware 12 to capture one or more images of the building 2. Processing circuitry of the drone 4 or computing system 8 may then analyze the one or more images (108). In various embodiments, the processing circuitry of the drone 4 or computing system 8 may analyze the image(s) received from the image capture hardware by executing one or more of the trained classification models, trained detection models, or trained segmentation models of the present disclosure to generate a model output.

[0116] The processing circuitry may report the model output (112). For example, the processing circuitry may be communicatively coupled to output hardware that is communicatively coupled to the processing circuitry. In these examples, the processing circuitry may be configured to output the model output via the output hardware, which may be a monitor, a speaker, a communication interface configured to relay the model input to another device, etc. As discussed above with respect to FIGS. 4A-4D and 6A-6D, the model output may indicate defect conditions and / or particular OoPSI(s) shown in the image(s).

[0117] Process 110 includes a determination of whether to mark the detected OoPSI using drone 4 (decision block 114). If the determination is to mark the detected OoPSI using drone 4 ("yes" branch of decision block 114), the control logic of drone 4 may cause drone 4 to mark the OoPSI (116), such as by using the techniques described above with respect to 13A and 13B. If the determination is not to mark the detected OoPSI using drone 4 ("no" branch of decision block 114), the site manager may optionally manually mark the detected OoPSI (118). The optional nature of the manual marking of the detected OoPSI is indicated by the dashed border of step 118 in FIG. 16.

[0118] Process 110 also includes a determination of whether to repair the detected OoPSI using drone 4 (decision block 120). If the determination is to repair the detected OoPSI using drone 4 ("yes" branch of decision block 120), control logic of drone 4 may cause drone 4 to repair the OoPSI (122), such as by using the techniques described above with respect to 14A-15. If the determination is not to repair the detected OoPSI using drone 4 ("no" branch of decision block 120), a site manager may optionally manually repair the detected OoPSI (124). The optional nature of manual repair of the detected OoPSI is indicated by the dashed border of step 124 in FIG. 16. In various embodiments, control logic of drone 4 may be configured to navigate drone 4 to an area surrounding the OoPSI and effect repair action in response to processing circuitry detecting a mark placed manually or by drone 4 by analyzing image(s) received from image capture hardware 12.

[0119] In some implementations, a software application executing on computing system 8 (which in these implementations is communicatively coupled to controller 6) autonomously identifies one or more targets on substrate 16 to be repaired via spraying by aerosol delivery system 102. The application can process video data of a video feed received from drone 4 (e.g., via image capture hardware 12 or other video capture hardware that drone 4 may include). For example, the application can identify a crack between two plywood boards and cause control logic of drone 4 to align drone 4 with an edge or end of the crack, activate aerosol dispenser system 90 to begin spraying, and move drone 4 along the crack until drone 4 reaches the opposite end of the crack, at which point the control logic can shut down aerosol delivery system 102 and stop spraying.

[0120] In another example, the application may identify a gap surrounding a pipe-substrate 16 joint and cause the control logic of the drone 4 to align the drone 4 with the edge of the crack, activate the aerosol dispenser system 90 to begin spraying, and move the drone 4 along a circular path that tracks the pipe-substrate 16 joint until the drone 4 has completely circled the joint, at which point the control logic may deactivate the aerosol delivery system 102 and stop spraying. In any of these examples, the application may identify the crack, pipe, or pipe joint by running a computer vision directed machine learning model trained using a dataset of multiple images of the substrate 16 at different distances, angles, lighting conditions, etc. Although described as "circular," it will be understood that the path of the drone 4 to repair the gap in the pipe joint may be an oval, an ellipse, or any other type of closed shape.

[0121] Computer vision processing can be performed on areas within the labeled bounding boxes around areas of interest. In one embodiment of a computer vision processing workflow of the present disclosure, an application running on computing system 8 can execute a trained machine learning algorithm to read video frames received from image capture hardware 12 and separate objects of interest from the image background (e.g., using color masking or other techniques), refine the mask (e.g., using morphological operations such as dilation, erosion, etc.), and detect one or more edges (e.g., using Canny edge detection).

[0122] In this example, the trained machine learning algorithm can erode the mask to remove outer edges, fit lines to edges (e.g., using a Hough line transform), filter out less relevant or irrelevant Hough lines (e.g., using DBSCAN clustering), and refine the intersections of the Hough lines with the mask edge(s). In this example, the trained machine learning algorithm can find the best-fit intersections (e.g., using k-means clustering), calculate the distance from the best-fit interaction point to the video center, and pass the variables to the control logic of the drone 4 via a wireless communication connection.

[0123] The variables can indicate crack initiation point, crack, angle, and other parameters that enable the control logic to navigate the drone 4 in a manner that enables the aerosol delivery system 102 to completely repair the detected crack(s). Although primarily described with respect to use of the aerosol delivery system 102, it will be appreciated that the computer vision processing aspects of the present disclosure also enable OoPSI marking (e.g., using the configurations shown in FIGS. 13A-13C) and / or OoPSI repair using adhesive delivery as shown by way of example in FIGS. 14A and 14B.

[0124] An exemplary procedure by which aspects of system 10 perform the repair procedure (whether using dispenser subassembly 72 or aerosol delivery system 102) is described below. In this example, the control logic of drone 4 can align drone 4 with the OoPSI to be repaired. The control logic can then actuate either dispenser subassembly 72 or aerosol delivery system 102 using any mechanism consistent with this disclosure, such as the optical toggle mechanism described above, the microcontroller-based mechanism described above, or the like, and aspects of system 10 can perform the computer vision procedure described above.

[0125] Based on the output of the computer vision procedure, the processing circuitry may determine whether the OoPSI angle (e.g., the crack angle) is within a predetermined range. If the crack angle is not within the predetermined range, the control logic may adjust the yaw of the drone 4 relative to the substrate 16 and re-run the computer vision procedure for evaluation of the OoPSI angle.

[0126] If the OoPSI is within the predetermined range (whether on the first iteration of the angle evaluation for an execution pass of the computer vision procedure or on a subsequent iteration), the processing circuitry can determine whether the end of the OoPSI (e.g., the crack end) is centered or substantially centered in a video frame or other image captured by the image capture hardware 12. If the end of the OoPSI is not centered or substantially centered in the frame, the control logic can adjust the pitch and / or roll of the drone 4 to move the drone 4 along the OoPSI, thereby aligning either the dispenser subassembly 72 or the aerosol delivery system 102 with the end of the OoPSI to initiate repairs at the appropriate location.

[0127] The processing circuitry may iteratively re-execute the computer vision procedure until the end of the OoPSI is substantially centered within the most recently captured frame via the image capture hardware. Upon a cycle of execution of the computer vision procedure that indicates that the OoPSI angle is within a predetermined range and that the end of the OoPSI is substantially centered within a frame captured by the image capture hardware 12, the control logic may shut down the dispenser subassembly 72 or the aerosol delivery system 102 and repair the OoPSI (e.g., using any of the optical toggle mechanisms described above, the microcontroller-based mechanisms described above, or any other actuation mechanism consistent with this disclosure).

[0128] In the following detailed description of the preferred embodiments, reference is made to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. The illustrated embodiments are not intended to be exhaustive of all embodiments in accordance with the present invention. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present invention. Therefore, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

[0129] Unless otherwise indicated, all numbers expressing size, quantities, and physical characteristics of features used in the specification and claims are to be understood as being modified in all instances by the term "about" or "approximately" or "substantially." Accordingly, unless specifically indicated to the contrary, the numerical parameters set forth in the above specification and appended claims are approximations that may vary depending upon the desired properties sought to be obtained by those of ordinary skill in the art utilizing the teachings disclosed herein.

[0130] As used in this specification and the appended claims, the singular forms "a," "an," and "the" include embodiments having plural referents unless the content clearly dictates otherwise. As used in this specification and the appended claims, the term "or" is used in its general sense including "and / or" unless the content clearly dictates otherwise.

[0131] It will be appreciated that, depending on the embodiment, certain acts or events of any of the methods described herein may be performed in a different order, or may be added, combined, or omitted altogether (e.g., not all of the acts or events described may be required for implementation of a method). Furthermore, in certain embodiments, acts or events may be performed simultaneously rather than sequentially, e.g., via multi-threaded processing, interrupt processing, or multiple processors.

[0132] The techniques described in this disclosure may be implemented at least in part in hardware, software, firmware, or any combination thereof. For example, various aspects of the described techniques may be implemented in one or more processors, including one or more microprocessors, CPUs, GPUs, DSPs, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or any other equivalent integrated or discrete logic circuitry, as well as any combination of such components. The term "processor" or "processing circuitry" may generally refer to any of the aforementioned logic circuits alone or in combination with other logic circuits, or other equivalent circuits. A control unit including hardware may also perform one or more of the techniques of this disclosure.

[0133] Such hardware, software, and firmware may be implemented within the same device or within separate devices to support various operations and functions described in this disclosure. Furthermore, any of the described units, modules, or components may be realized together or separately as discrete but interoperable logical devices. The depiction of different features as modules or units is intended to highlight different functional aspects and does not necessarily imply that such modules or units must be realized by separate hardware or software components. Rather, functionality associated with one or more modules or units may be performed by separate hardware or software components or may be incorporated in common or separate hardware or software components.

[0134] The techniques described in this disclosure may also be embodied or encoded in a computer-readable medium, such as a computer-readable storage medium containing instructions. The instructions embedded or encoded in the computer-readable storage medium may, for example, cause a programmable processor or other processor to perform a method when executed. The computer-readable storage medium may include random access memory (RAM), read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), flash memory, hard disk, CD-ROM, floppy disk, cassette, magnetic medium, optical medium, or other computer-readable medium.

[0135] Various embodiments have been described. These and other embodiments are within the scope of the following claims.

Claims

1. polarized camera hardware configured to capture polarized images of the tape applied to the substrate; a memory communicatively coupled to the polarization camera hardware and configured to store the polarization image; a processing circuit communicatively coupled to the memory, Analyzing the polarized image according to a trained classification model; detecting misapplication of the tape applied to the substrate based on the analysis of the polarized image according to the trained classification model. A processing circuit configured as above; A system comprising:

2. The system of claim 1 , wherein the trained classification model is a trained neural network model.

3. The system of claim 1 , wherein the polarization camera hardware is embedded in a mobile computing device.

4. The system of claim 1 , wherein the polarization camera hardware is communicatively coupled to a mobile computing device.

5. The system of claim 3 , wherein the mobile computing device comprises one of a smartphone, a tablet computer, or a wearable computing device.

6. The system of claim 1 , wherein the polarization camera hardware is communicatively coupled to a drone.

7. The system of claim 1 , wherein the polarization camera hardware is integrated into a drone.

8. 10. The system of claim 1, further comprising output hardware communicatively coupled to the processing circuitry, the processing circuitry further configured to output, via the output hardware, a model output indicative of the misapplication of the tape applied to the substrate.

9. 10. The system of claim 1, wherein the misapplication is associated with at least one of fishmouth wrinkles or wrinkles in the tape applied to the substrate.

10. 2. The system of claim 1, wherein the trained classification model is configured to implement one or more of overall image classification, sub-image classification, object detection, or image segmentation for the image.

11. The system of claim 1 , wherein the substrate is a building envelope layer.

12. The system of claim 1 , wherein the polarized image indicates separation of focal plane (DoLP) data for the tape applied to the substrate.

13. capturing a polarized image of the tape applied to the substrate with polarized camera hardware; analyzing, by a processing circuit communicatively coupled to the image capture hardware, the polarized image according to a trained classification model; detecting, by the processing circuitry, a misapplication of the tape applied to the substrate based on the analysis of the polarized image according to the trained classification model; A method comprising:

14. The method of claim 13 , wherein the trained model is a trained neural network model.

15. The method of claim 13 , wherein the polarization camera hardware is integrated into a mobile computing device.

16. The method of claim 13 , wherein the polarization camera hardware is integrated into a drone.

17. 14. The method of claim 13, further comprising outputting, by the processing circuitry, via output hardware communicatively coupled to the processing circuitry, a model output indicative of the misapplication of the tape applied to the substrate.

18. 14. The method of claim 13, wherein the misapplication is associated with at least one of fishmouth wrinkles or wrinkles in the tape applied to the substrate.

19. The method of claim 13 , wherein the trained model is configured to implement one or more of overall image classification, sub-image classification, object detection, or image segmentation on the image.

20. The method of claim 13 , wherein the substrate is a building envelope layer.

21. The method of claim 13 , wherein the polarized image shows separation of focal plane (DoLP) data for the tape applied to the substrate.

22. a means for capturing a polarized image of the tape applied to a substrate; means for analyzing the polarized image according to a trained model; means for detecting misapplication of the tape applied to the substrate based on the analysis of the polarized image according to the trained model; An apparatus comprising:

Images