Automated aerial imagery analysis for building segmentation and roof classification

Abstract

A method is described herein comprising receiving an aerial image of a roof structure, receiving image information of the aerial image, wherein the image information includes pixel length of the image, pixel height of the image, and altitude of the sensor capturing the image, using a trained model to analyse the aerial image and predict linear features of the roof structure, converting each linear feature into a geometric representation, wherein each geometric representation comprises an ordered list of two dimensional pixel coordinates in cartesian space, computing sensor parameters using the image information, wherein the sensor parameters comprise sensor width and sensor height, converting the geometric representations of the linear features into physical length dimensions using the sensor parameters and the image information, and computing surface area of the roof structure using the geometric representations of the linear features and their corresponding physical length dimensions.

Description

[0001] Docket No. 2954678-00004-W01

[0002] Filed January 22, 2026 AUTOMATED AERIAL IMAGERY ANALYSIS FOR BUILDING SEGMENTATION AND ROOF CLASSIFICATION

[0003] RELATED APPLICATIONS

[0004] This application claims the benefit of US Application No. 63 / 748,307, filed January 22, 2025.

[0005] TECHNICAL FIELD

[0006] The present disclosure relates to software implemented roof sketch generation technologies, under an embodiment.

[0007] INCORPORATION BY REFERENCE

[0008] Each patent, patent application, and / or publication mentioned in this specification is herein incorporated by reference in its entirety to the same extent as if each individual patent, patent application, and / or publication was specifically and individually indicated to be incorporated by reference.

[0009] SUMMARY OF THE INVENTION

[0010] A method is described herein comprising under an embodiment receiving an aerial image of a roof structure, receiving image information of the aerial image, wherein the image information includes pixel length of the image, pixel height of the image, and altitude of the sensor capturing the image, using a trained model to analyse the aerial image and predict linear features of the roof structure, wherein the linear features comprise eaves, ridges, hips, valleys, and rakes, converting each linear feature into a geometric representation, wherein each geometric representation comprises an ordered list of two dimensional pixel coordinates in cartesian space, computing sensor parameters using the image information, wherein the sensor parameters comprise sensor width and sensor height, converting the geometric representations of the linear features into physical length dimensions using the sensor parameters and the image information, and computing surface area of the roof structure using the geometric representations of the linear features and their corresponding physical length dimensions.Docket No. 2954678-00004-W01

[0011] Filed January 22, 2026 In embodiments, the converting the geometric representations of the linear features comprises determining pixel length of each geometric representation.

[0012] In embodiments, the converting the geometric representations of the linear features comprises determining Euclidean distance between consecutive points of each geometric representation.

[0013] In embodiments, the converting the geometric representations of the linear features comprises determining ground sample distance parameters for the linear features of the roof structure.

[0014] In embodiments, the converting the geometric representations of the linear features comprises multiplying pixel length of each geometric representation by its respective ground sample distance parameter.

[0015] In embodiments, the computing the surface area comprises representing the geometric representations as an undirected planar graph.

[0016] In embodiments, the computing the surface area comprises traversing the undirected planar graph to identify discrete polygonal components of the undirected planar graph.

[0017] In embodiments, the computing the surface area comprises determining pixel perimeter of the graph by summing the pixel length of the geometric representations.

[0018] In embodiments, the computing the surface area comprises determining pixel area of each discrete polygonal component.

[0019] In embodiments, the computing the surface area comprises determining total pixel area of the undirected planar graph by summing the pixel area of the discrete polygonal components.

[0020] In embodiments, the computing the surface area comprises determining total physical perimeter of the undirected planar graph.

[0021] In embodiments, the determining the total physical perimeter of the undirected planar graph comprises summing the physical length dimensions of the geometric representations.

[0022] In embodiments, the computing the surface area comprises generating a ratio parameter equal to the physical length perimeter of the undirected planar graph divided by the pixel perimeter of the undirected planar graph.

[0023] In embodiments, the computing the surface area comprises multiplying total pixel area of the undirected planar graph by the ratio parameter.Docket No. 2954678-00004-W01

[0024] Filed January 22, 2026 BRIEF DESCRIPTION OF THE FIGURES

[0025] Figure 1 shows an aerial image, under an embodiment.

[0026] Figure 2 shows an aerial image rendered in grayscale, under an embodiment.

[0027] Figure 3 shows a coded aerial image, under an embodiment.

[0028] Figure 4 shows a CVAT screen shot illustrating coded annotations overlaid on a top down view of a building structure, under an embodiment.

[0029] Figures 5A and 5B show an original aerial image and predicted mask, under an embodiment.

[0030] Figures 6A and 6B show an original aerial image and predicted mask, under an embodiment.

[0031] Figures 7A and 7B show an original aerial image and predicted mask, under an embodiment.

[0032] Figures 8A and 8B show an original aerial image and predicted mask, under an embodiment.

[0033] Figure 9 shows eave pixels and corresponding length, under an embodiment.

[0034] Figures 10A-10C show computation of a diagonal length, under an embodiment.

[0035] Figure 11 shows an example of line splitting, under an embodiment.

[0036] Figure 12A shows an example of roof attribute total area and feature dimensions, under an embodiment.

[0037] Figure 12B shows an example of roof attribute dimensions, under an embodiment.

[0038] DETAILED DESCRIPTION

[0039] Automating roof sketch generation for creating roof sketches with machine learning involves building a model that process aerial or satellite imagery to automatically produce precise roof outlines. For this project, ACT (or the Company) leverages a U-Net model to achieve this as further described below.

[0040] The Company uses a Google Satellite API to access relevant property information and imagery. The Company gathers high-resolution aerial imagery and property details directly from Google Earth, ensuring the accuracy and completeness of the dataset for model training and evaluation.

[0041] Figure 1 shows an aerial image, under an embodiment.Docket No. 2954678-00004-W01

[0042] Filed January 22, 2026 Under an alternative embodiment, the Company acquires images through VExcel, Bing or Google Satellite API. Under yet another embodiment, aerial images may be obtained through drone images.

[0043] Once the aerial images are collected, they are processed to ensure they can be used effectively by our models. This preprocessing ensures that they all have the same size, colors, and quality. This helps to make all the images uniform and easier to analyze accurately for the model. On a daily basis, we download the image from Google Earth using the address we have in our database. The method searches for the roof tops using our orders database table which has an address for each order. At one location there may be multiple roof tops, so the approach crops to that part of the image which contains the correct roof top image. After that we change the format of image from RGB to Grayscale and then we change the dimension of images to 256 x 256 (height x width). The Company uses a Grayscale image of dimension 256*256 (see Figure 2) for model training, under an embodiment.

[0044] After preprocessing the images, the Company prepares the data for model training by uploading them to a browser based Computer Vision Annotation Tool (CVAT) for annotation. This process involves labelling various roof attributes using a color-coded scheme (or any type of scheme which clearly distinguishes annotations). Once annotated, the data is exported in XML format. The input images, along with their corresponding XML files, are then used to generate the segmentation masks required for the model training.

[0045] As indicated, the Company uses CVAT (Computer Vision Annotation Tool) browser based version for annotating different attributes of the roof. We have a trained team which have their email id registered with CVAT.

[0046] On a daily basis after preprocessing the image (described in above step) trained users upload it on CVAT separately and then manually annotate the uploaded images.

[0047] Once annotated, annotated images are exported in (CVAT for image 1.1) format. CVAT for image 1.1 is one of the formats provided by CVAT to export the “annotations” to an XML file which contains the co-ordinates for the attributes within the image which is later used to create the mask of that roof top. After this step all of the raw images and their annotations are stored in a “one drive” location.

[0048] The annotation process is automated under an embodiment. This approach identifies rooftops of interest from a database using latitude and longitude information. Such images mayDocket No. 2954678-00004-W01

[0049] Filed January 22, 2026 be downloaded directly into azure cloud storage using google satellite API. (Note that database images are on zoom level 20-22 of google satellite API). After that a pre-trained model (i.e. trained using the methods described herein) hosted on a local server predicts a mask which is uploaded to CVAT for annotation.

[0050] The xml data exported from CVAT contains information about roof segment (and its location in the image) along with image name. The sample xml file below shows collected information. After exporting to XML, we run a python program to create one image per xml file. Or rather, multiple roof top images are annotated at once. CVAT then exports the annotations of all the images into one XML file. Then the python program parses the XML data to create one XML file for each image. The XML file and image are used to create a mask which is fed into training data.

[0051] Figure 3 shows a coded image which corresponds to XML data below.

[0052] {

[0053] "name": "roof top",

[0054] "id": 2901002,

[0055] "color": "#968800",

[0056] "type": "any",

[0057] "attributes": []

[0058] },

[0059] {

[0060] "name": "ridge",

[0061] "id": 2901003,

[0062] "color": "#3d3df5",

[0063] "type": "any",

[0064] "attributes": []

[0065] },

[0066] {

[0067] "name": "hip",

[0068] "id": 2901004,

[0069] "color": "#32b7fa",

[0070] "type": "any",

[0071] "attributes": []

[0072] },

[0073] {

[0074] "name": "valley",

[0075] "id": 2901005,

[0076] "color": "#e025e0",Docket No. 2954678-00004-W01

[0077] Filed January 22, 2026 "type": "any",

[0078] "attributes": []

[0079] },

[0080] {

[0081] "name": "rake",

[0082] "id": 2901006,

[0083] "color": "#fa3253",

[0084] "type": "any",

[0085] "attributes": []

[0086] },

[0087] {

[0088] "name": "eave",

[0089] "id": 2901007,

[0090] "color": "#24b353",

[0091] "type": "any",

[0092] "attributes": []

[0093] }

[0094] Figure 3 actually shows annotations identified by line type. As indicated in the code above, color coding may also be used as opposed to line type. (Figure 3 does not show a rake. However, it should be noted that a rake would appear from a top down perspective as an eave with some midpoint).

[0095] Figure 4 shows a CVAT screen shot illustrating annotations overlaid on a top down view of the building structure shown in Figure 3, under an embodiment.

[0096] The Company utilizes the U-Net architecture for the image segmentation task. This model functions as a sophisticated algorithm, learning from labelled images to detect and classify essential features. We meticulously design and configure the U-Net model to ensure it accurately captures the relevant details. Its architecture is optimized with carefully selected layers and connections to enable efficient image processing and precise segmentation. Additionally, we focus on enhancing both accuracy and performance, fine-tuning the model’s parameters and structure to efficiently process large datasets while delivering reliable predictions.

[0097] Model Configuration:

[0098] 1. Input Layer.

[0099] —The model uses images with a 256*256 input size.

[0100] —It uses a batch processing mechanism, where images are processed batches of 4 in size.Docket No. 2954678-00004-W01

[0101] Filed January 22, 2026 2. Encoder (Down sampling Path):

[0102] —The down sampling is achieved through Convolutional Layers (Conv2D) followed by MaxPooling layers.

[0103] —Each convolution operation uses filters to extract spatial features. The number of filters is doubled after each downsampling step. It started from 64 and goes up to 256.

[0104] 3. Decoder (Upsampling Path):

[0105] —The decoder uses UpSampling2D layers to increase the spatial dimensions, followed by concatenation with feature maps from the corresponding downsampling layers. This allows the model to retain fine-grained details from earlier layers.

[0106] —Similar to the encoder, Conv2D layers are applied after each upsampling operation from 64 to 256.

[0107] 4. Output Layer:

[0108] —The output layer uses a final convolution to predict the class for each pixel in the image, representing different categories (e.g., ridges, eaves, etc.) including background.

[0109] —The number of filters in the output layer corresponds to the number of classes which is 6 including background.

[0110] 5. Loss Function:

[0111] —We use categorical_crossentropy as loss function.

[0112] 6. Optimizer:

[0113] —The model uses the Adam optimizer.

[0114] 7. Training Setup:

[0115] —Batch Size: The model is trained using a batch size of 4 by default, meaning 4 images are processed together in each iteration.

[0116] —Epochs: We are running 50 epochs.

[0117] Fine-tuning:

[0118] Fine-tuning typically involves:

[0119] —Adjusting the number of filters in Conv2D layers based on dataset complexity.

[0120] —Modifying the loss function to handle imbalanced classes or specific performance goals.

[0121] —Optimizing the learning rate schedule for faster convergence.

[0122] Platform for Running the Model:Docket No. 2954678-00004-W01

[0123] Filed January 22, 2026 The model is implemented using TensorFlow and Keras, both highly scalable libraries that can leverage GPU / TPU for training. TensorFlow allows the use of distributed computing, making it efficient for large datasets. U-Net models are computationally intensive, so running on GPUs (NVIDIA Tesla or similar) significantly speeds up training, under an embodiment.

[0124] Efficiency Considerations:

[0125] To improve efficiency:

[0126] Batch Size: The batch size of 4 is small, reducing memory overhead but potentially slowing down training.

[0127] Early Stopping: Monitoring validation loss to stop training when overfitting begins.

[0128] Model Pruning / Quantization: These techniques are applied post-training to reduce the model’s size and increase inference speed.

[0129] Current platform:

[0130] The model runs on Kaggle in free tier for purposes of training and running the model, under an embodiment.

[0131] Figures 5A and 5B show an original aerial image and predicted mask, under an embodiment.

[0132] Figures 6A and 6B show an original aerial image and predicted mask, under an embodiment.

[0133] Figures 7A and 7B show an original aerial image and predicted mask, under an embodiment.

[0134] Figures 8A and 8B show an original aerial image and predicted mask, under an embodiment.

[0135] The method described herein calculates Ground Sample Distance (GSD) in feet per pixel using key parameters such as image capture altitude, image width and height, focal length, and sensor dimensions. While Google Earth Pro provides most of this data, it lacks sensor width and height. To resolve this, we used a trial-and-error approach and found that sensor dimensions of approximately 30x30 produced accurate results, which we validated across multiple cities, including Florida, Miami, California, and Oklahoma.

[0136] Ground Sample Distance (GSD) is a term used to describe the amount of ground space that one pixel in an aerial or satellite image covers. It is commonly used in aerial and satellite imagery to measure the spatial resolution of the imagery. It is calculated as follows...Docket No. 2954678-00004-W01

[0137] Filed January 22, 2026 gsd_width_meters_perj>ixel=(altitude*sensor_width_mm) / (focal_length* image_width) gsd_height_meters_per_pixel=(altitude*sensor_height_mm) / (focal_length*

[0138] image_height) gsd_meters_per_pixel=(gsd_width_meters_per_pixel+gsd_height_meters_per_pixel) / 2 gsd_feet_per_pixel = gsd_meters_per_pixel * 3.28084

[0139] • Altitude: This refers to the height of viewpoint (camera) above the ground or sea level as we view the earth in Google Earth Pro.

[0140] • Focal Length: This refers to the distance between the camera lens and the camera sensor when the subject is in focus.

[0141] • Sensor Width and Height: This refers to the physical dimensions of the camera's imaging sensor used to capture the aerial or satellite images.

[0142] • Image Width and Height: This refers to actual dimensions of captured image. After establishing the GSD in feet per pixel, the method described herein calculated the length of each roof attribute by counting the corresponding pixels and multiplying by the GSD. This method works well for straight lines but is less effective for diagonal lines.

[0143] Under an embodiment, the method computes GSD in closed form when sensor width and height are known. As indicated above, an alternative embodiment implements a trial-and-error approach. This alternative found that sensor width and height dimensions of approximately 30x30 produced accurate results, which was validated across multiple cities, including Florida, Miami, California, and Oklahoma.

[0144] We ran some trial and error on images of 10 US cities to find an average for sensor height and data.

[0145] Figure 9 illustrates computed length of eaves (shown in yellow). After establishing the GSD in feet per pixel, the method calculates the length of each roof attribute by counting the corresponding pixels and multiplying by the GSD. This method works well for straight lines but is less effective for diagonal lines.

[0146] Diagonal lines tend to cover more pixels than straight horizontal or vertical lines in an image because of the way pixels are arranged in a grid pattern on a digital display or image.

[0147] A horizontal line or vertical line will align directly along the rows or columns of pixels, covering only those that lie directly along that line.Docket No. 2954678-00004-W01

[0148] Filed January 22, 2026 When a line is diagonal, it has to cross over both rows and columns as it moves across the image. As a result, a diagonal line will typically touch more pixels than a straight line of the same length because it does not align perfectly with the pixel grid. Each pixel represents a square, under an embodiment. So, foot per pixel accurately represents length of one side of the pixel. The foot per pixel parameter does not apply to the diagonal of the pixel itself. Diagonal lines are oblique and not a straight line like eaves or ridge as seen in Fig. 3. However, hips and valleys are oblique. And diagonal lines contain more pixels than straight lines.

[0149] To address this, we drew a virtual perpendicular line 1002 from the point where two hips meet down to the corresponding eave 1004. By treating the eave as the base and using the perpendicular line, we applied the Pythagorean theorem to accurately calculate the length of the hip 1006.

[0150] Figures 10A-10C illustrates computation of a diagonal line using the method described above.

[0151] Note that images and predicted masks are treated as flat projections. The distance from aerial camera to the particular roof structure is large relative to height of roof attributes with respect to ground. Accordingly, the flat image projection is sufficient for determining length of roof attributes without accounting for height of roof attributes or their slope relative to ground.

[0152] The following provides additional technical disclosure related to automated aerial imagery analysis for building segmentation and roof classification, under an embodiment.

[0153] 1. High-Level Project Objective

[0154] The systems and method described herein accurately detect roof attributes from aerial imagery and compute their real-world geometric measurements (lengths and areas) in a reliable, scalable, and explainable manner.

[0155] Key roof attributes include (but are not limited to): - Eaves - Ridges - Hips - Valleys -Rakes. Accuracy at the individual attribute level is critical because downstream calculations (costing, material estimation, reports) depend on precise per-segment measurements rather than coarse aggregated values.

[0156] 2. Semantic Segmentation (UNet)

[0157] An initial implementation is based on semantic segmentation using a UNet architecture, under an embodiment. In this approach each pixel in the image was classified into a roof-relatedDocket No. 2954678-00004-W01

[0158] Filed January 22, 2026 class (eg., eave, ridge, valley). - All pixels belonging to the same class were grouped together, regardless of whether they represented multiple physical instances.

[0159] Characteristics of Semantic Segmentation

[0160] 1 No Instance Separation

[0161] If a roof contained multiple eaves, all eave pixels were merged into a single mask. Accordingly, eave_l, eave_2, eave_3, etc. are indistinguishable.

[0162] 2. Length Aggregation

[0163] Since multiple physical segments were merged, length computation were performed at a class level, not per physical roof edge.

[0164] 3. Downstream Measurement Errors

[0165] Any error or noise in one part of the mask affected the total measurement for the entire class. Instance-based segmentation offers a slightly different approach to automated aerial imagery analysis for building segmentation and roof classification

[0166] 3. Architectural Shift: Instance-Based Segmentation

[0167] To enable independent measurement of each roof attribute, the system was redesigned to use instance-based segmentation, where: - Each physical roof element is detected as a separate instance - Example: eave_l, eave_2, eave_3 are treated independently. This shift allows: -Accurate per-instance geometry extraction - Independent length computation - Robust handling of complex roofs with repeated attributes.

[0168] In the semantic segmentation approach described above, roof features such as eaves, ridges, rakes, valleys, hips, and similar structures were treated using a purely semantic labelling approach. If multiple instances of the same feature type were present on a roof (for example, multiple ridges or multiple eaves), all of them were labelled under a single class name (e.g., ridge or eave. While this allowed general feature identification, it did not distinguish individual instances within the same category.

[0169] In the updated training process, the labelling strategy has been enhanced to be instance-aware across all roof features. When multiple instances of a feature are present, each is assigned a unique instance label such as eave_l, eave_2, ridge_l, ridge_2, valley_l, hip_1, etc., while still being grouped under their respective semantic categories (eaves, ridges, rakes, valleys, hips).

[0170] The overall training process remains consistent with the methodology described with respect to semantic segmentation. The system continues to rely on supervised learning using the same types of image data, the same subject matter labels, and the same quality-controlledDocket No. 2954678-00004-W01

[0171] Filed January 22, 2026 annotation process The purpose of the training —to identify and delineate roof-related features — has not changed.

[0172] The only substantive modification is a change in the underlying model architecture used during training. The original implementation utilized a model that labelled all image areas simultaneously at the pixel level. The current implementation uses a model that first identifies individual structural components within an image and then generates precise outlines for each identified component.

[0173] It changes how the model internally processes the images in order to improve accuracy, particularly in cases involving complex or overlapping structures.

[0174] The prior implementation used a U-Net -based semantic segmentation architecture; the current implementation uses an instance-based segmentation model implemented with Detectron2

[0175] Model description:

[0176] Task type: Instance segmentation (object-level detection with per-instance masks). Backbone: Convolutional neural network (e.g., ResNet family) for feature extraction. Multi-scale features: Roof components can appear large or small in images, FPN allows the model to understand objects at different sizes simultaneously.

[0177] Region proposal Region Proposal Network (RPN): Instead of scanning the entire image in detail, the model first suggests regions that are likely to contain roof components.

[0178] Heads: Once regions are processed, the model uses three specialized sub-models (“heads”) to make final predictions.

[0179] a. Classification Head

[0180] Decides what type of roof component each region represents.

[0181] b. Mask Head

[0182] Generates a pixel -level outline for each detected roof component.

[0183] Outputs: For each detected instance, the model produces a class label, confidence score and a pixel-level mask.

[0184] Training: Supervised learning using annotated instances, losses include classification loss, box regression loss, and mask loss; standard SGD-based optimization is used.Docket No. 2954678-00004-W01

[0185] Filed January 22, 2026 This model is implemented using a well-known, off-the-shelf computer vision model that is widely available in open-source libraries. But we are Fine-Tuning this with our own custom roof dataset.

[0186] Plain-language explanation of the model

[0187] Step I: Understanding the image (Backbone)

[0188] The model first uses a pretrained image reader (a ResNet-style convolutional neural network).

[0189] One can think of this as converting the raw photo into a set of visual clues, such as edges, corners, and textures — like how the human eye notices shapes before recognizing objects Example:

[0190] Before identifying a roof ridge, the model first notices long straight lines and sharp boundaries.

[0191] Step 2: Handling big and small roof parts (multi-scale features with FPN)

[0192] Roof components can be:

[0193] • Very large (e g., a long ridge line), or

[0194] • Very small (e.g., a short hip).

[0195] The Feature Pyramid Network (FPN) allows the model to look at the image at multiple zoom levels at the same time

[0196] This ensures small objects are not missed and large objects are not broken apart.

[0197] Example:

[0198] A valley running across half the roof and a small hip on the centre, both detected reliably. Step 3: Finding ‘‘areas of interest” (Region Proposal Network)

[0199] Instead of examining every pixel in detail, the model first suggests likely regions where roof components might exist.

[0200] This is similar to a human quickly scanning a photo and saying:

[0201] “These areas look important— let’s inspect them more closely.”

[0202] These suggested regions are called region proposals.

[0203] Step 4: Making final decisions (Prediction heads)

[0204] Each proposed region is passed to three small decision-makers (called “heads”):

[0205] a. Classification Head

[0206] Determines what kind of roof component the region containsDocket No. 2954678-00004-W01

[0207] Filed January 22, 2026 Example:

[0208] “This region looks like a ridge, not a valley.”

[0209] b. Box Regression Head

[0210] Adjusts the region boundaries so the object is tightly framed.

[0211] Example:

[0212] '‘The ridge extends slightly farther left than initially estimated.”

[0213] c. Mask Head

[0214] Draws a precise pixel-by-pixel outline of the roof component.

[0215] Example:

[0216] Instead of a rough rectangle, the model produces an exact ridge shape that follows the roof line.

[0217] Final output (What the model produces)

[0218] For each detected roof component, the model outputs:

[0219] • A label (e.g., ridge, hip, valley),

[0220] • A confidence score (how sure the model is), and

[0221] • A pixel-level mask showing the exact shape and location.

[0222] This makes the output suitable not only for visualization, but also for measurement, quantity estimation, and downstream analysis.

[0223] Training process (How the model learns)

[0224] The model is trained using example images where roof components are already marked by humans

[0225] During training, it learns by minimizing three main types of error:

[0226] 1. Classification loss - was the object labeled correctly?

[0227] 2. Bounding box loss - was the object location accurate?

[0228] 3. Mask loss - does the predicted shape match the true shape?

[0229] In summary, the change is a model architecture transition from pixel-wise semantic segmentation (U-Net) to instance-based segmentation using Detectron2. The training objective and labelled concepts remain the same; only the modelling approach has been updated.

[0230] This change enables the model to learn and predict each roof feature as a distinct instance, improving separation between adjacent or overlapping structures. It also provides more accurate geometric outputs, which directly supports downstream processes such as, individualDocket No. 2954678-00004-W01

[0231] Filed January 22, 2026 length calculation, measurement verification, and property-level review. Overall, the revised training approach improves precision, clarity, and scalability across all roof feature types.

[0232] Each detected instance is represented by its polyline - Instances are processed independently throughout the pipeline as further described below.

[0233] Each detected instance corresponds to a linear structural feature, and all instances are therefore linear in nature.

[0234] During model output, the instance-segmentation model produces a pixel-level mask for each detected instance. Although the internal representation is a filled mask, this mask is used only as an intermediate step

[0235] For each detected instance:

[0236] • The system identifies the first and last occurrences of the instance pixels along the structure

[0237] • The corresponding pixel coordinates are extracted.

[0238] • These ordered coordinates are then converted into a polyline representation.

[0239] 4. Instance Geometry Extraction and Polyline Generation

[0240] Once predictions are obtained from the instance-based segmentation model, the system transitions from pixel-level masks to vector-based geometric representations that can be measured, edited, and audited.

[0241] 4.1 Coordinate Extraction from Instance Predictions

[0242] For each predicted instance (e g., individual eave, ridge, hip, valley):

[0243] • The instance mask is isolated from the model output

[0244] • Boundary pixels corresponding to that instance are extracted

[0245] • These pixels are converted into ordered coordinate sets in two dimensional (x-y) image (pixel) space

[0246] As indicated above, instances are converted into two-dimensional (x-y) pixel space. At this stage, each roof attribute exists as a set of raw coordinates, still closely tied to the raster prediction.

[0247] 4.2 Conversion of Coordinates to Polylines

[0248] The extracted coordinates are then simplified and converted into polylines, where:

[0249] • Each polyline represents a single physical roof attribute instance

[0250] • Polylines are defined as an ordered list of points (start → end)Docket No. 2954678-00004-W01

[0251] Filed January 22, 2026 A polyline is the connected, start -to-end representation of an ordered set of coordinates. The term '■‘polyline” refers to how the ordered coordinates are used and interpreted as a single linear feature. Suppose there are two points in two-dimensional space: (a, b) and (c, d). Individually, these are simply two separate points in the image. When these two points are defined as a polyline from (a, b) to (c, d), they are interpreted as a single connected line segment spanning from (a, b) to (c, d)

[0252] • Noise and redundant intermediate points are reduced where possible. Noise is defined as duplicate instance predictions occurring at the same pixel coordinates. It is not the presence of intermediate points, but the repetition of identical instances. Noise is reduced by identifying and removing overlapping or duplicate instance predictions during post¬ processing.

[0253] This conversion provides a clean vector representation that enables: - Precise length measurement - Topological reasoning (connections, intersections, loops) - Downstream area computation

[0254] 4.3 Manual Polyline Review and Correction

[0255] Despite high model accuracy, automated predictions may occasionally deviate from the desired or visually correct roof geometry due to: - Occlusions - Low-contrast imagery - Complex roof structures - Model uncertainty at intersections

[0256] To address this, the system supports manual polyline modification:

[0257] • Users can adjust polyline vertices directly in the frontend

[0258] • Polylines can be reshaped, extended, shortened, or snapped to correct locations

[0259] • These edits ensure that vectors precisely match the intended roof geometry

[0260] 4.4 Validation and Propagation of Edits

[0261] After manual correction:

[0262] • Updated polylines are sent back to the backend

[0263] • Lengths are recalculated using the corrected geometry

[0264] • Each roof attribute length is reviewed individually to ensure it aligns with expected real-world dimensions.

[0265] • If any property length appears inconsistent, it is manually adjusted by refining the corresponding polyline

[0266] • Intersection splitting and area computation are re-run on the modified dataDocket No. 2954678-00004-W01

[0267] Filed January 22, 2026 This guarantees that all downstream calculations (lengths, faces, areas) are based on the final, user-validated geometry rather than raw model output.

[0268] It should be noted that this manual correction feature is entirely optional and that aerial imagery analysis for building segmentation and roof classification is entirely automated under one embodiment.

[0269] 4.5 Benefits of the Hybrid Approach

[0270] By combining: - Instance-based model predictions - Deterministic geometric processing -Manual correction capability

[0271] The system achieves: - High automation where possible - Full control and correctness where required - Auditability of every geometric decision

[0272] This hybrid workflow is essential for production-grade roof measurement systems.

[0273] Once instance segmentation is completed, the following steps are applied:

[0274] 1. Coordinate Extraction

[0275] For each detected instance, the boundary or centerline coordinates are extracted in pixel space.

[0276] 2. Pixel-Domain Length Calculation

[0277] The Euclidean distance formula is applied between consecutive coordinate points: distance_px = sqrt((x2 - x1)^2 + (y2 - y1)^2)

[0278] 3. Total Pixel Length per Instance

[0279] All segment distances belonging to a single instance are summed to produce a final pixel length. At this stage, measurements are still in pixel units and must be converted to real-world units.

[0280] 5. Ground Sample Distance (GSD) - Revised Methodology

[0281] 5.1 Motivation for Revising GSD Calculation

[0282] Previously, GSD estimation relied on trial-and-error assumptions for sensor dimensions, which introduced instability and reduced reproducibility. To improve accuracy and consistency, the GSD computation process was re-engineered using known properties of Google Earth imagery.

[0283] 5.2 Derving Sensor Dimensions

[0284] Key observations: - Google Earth imagery provides a scale legend that varies with altitude - The saved image dimensions (width and height in pixels) are known - Altitude metadata is available.

[0285] Using these known values, the sensor dimensions were reverse engineered and determined to be:Docket No. 2954678-00004-W01

[0286] Filed January 22, 2026 Sensor width: 41 mm

[0287] • Sensor height: 23 mm

[0288] These two parameters now apply for all Google Earth images.

[0289] Sensor width and sensor height are calculated as follows:

[0290] H x Sensor Width

[0291] — _

[0292]

[0293] f * Image Width (pixels)

[0294] Where,

[0295] H Altitude,

[0296] And f = focal length

[0297] First, we manually calculate the Ground Sample Distance (GSD)

[0298] An image is captured from Google Earth with a resolution of 1920 x 1080 pixels, and the altitude at the time of capture is noted as 51 meters.

[0299] In the bottom-right corner of the image, there is a scale legend indicating a ground distance of 10 meters. This distance is convened to feet:

[0300] 10m = 10*3.28084 feet = 32.8084 feet

[0301] Next, we determine the number of pixels corresponding to this 10-meter scale.

[0302] Two reference points are selected at the extreme ends of the scale legend;

[0303] • Leftmost point: (1605, 1076)

[0304] • Rightmost point: (1916, 1076)

[0305] The pixel distance between these two points is calculated as:

[0306] 1916-1605 = 310 pixels

[0307] Using this value, the GSD (feet per pixel) is computed by dividing the real-world distance by the pixel distance:

[0308] GSD = 32.8084 / 310 = 0.10583 feet / pixel

[0309] Now that the GSD is known, the remaining parameters are:

[0310] • Focal length: 35 mm (as provided by Google Earth)

[0311] • Altitude: 51 meters = 167.32 feet

[0312] • Image width: 1920 pixels

[0313] Substituting the known values and rearranging to solve for sensor width:

[0314] 0.10583 - [167.32 *(Sensor Width)] / 35 * 1920

[0315] Sensor Width = (0 10583 * 35 *1920) / 167.32Docket No. 2954678-00004-W01

[0316] Filed January 22, 2026 Sensor Width = 42.5 (rounded to 43 mm.)

[0317] A similar approach is used to compute sensor height using the following equations.

[0318] H x Sensor Height

[0319] GSD = - - -

[0320]

[0321] f * Image Height (pixels)

[0322] But since Google Earth does not provide vertical scale legend we calculate GSD a little differently.

[0323] In a Google Earth image, one can measure the length of a vertical feature using Google Earth ruler. Now we calculate the pixel count between endpoints of the vertical feature.

[0324] The method saves the image from Google Earth which includes its properties (including the focal length of 35mm.

[0325] These sensor width and height values are now treated as fixed and consistent across the pipeline.

[0326] 6. Direction-Aware GSD Calculation

[0327] Instead of using a single GSD value for all measurements, the system now applies direction-aware scaling, which significantly improves real-world accuracy.

[0328] 6.1 Horizontal GSD

[0329] Used for predominantly horizontal roof attributes (e.g., horizontal eaves): gsd_horizontal_m = (altitude_m * sensor_width_mm / focal_length_mm) / image_width_px 6.2 Vertical GSD

[0330] Used for predominantly vertical roof attributes:

[0331] gsd_vertical_m = (altitude_m * sensor_height_mm / focal_length_mm) / image_height_px These are the same equations as used for calculating sensor width and height So now we know the sensor width and height, and the only variable left is the altitude which is provided by Google Earth for each image For every different altitude, GSD (horizontal, vertical,, and diagonal) will be different.

[0332] In capturing the image from Google Earth, the method captures the altitude While the pixel width and height of the imagery can vary conceptually, all images are consistently exported from Google Earth at a fixed resolution of 1920 x 1080. For this reason, the image dimensions are not treated as variables in the pipeline.

[0333] gsd horizontal

[0334] Ground Sample Distance in the horizontal directionDocket No. 2954678-00004-W01

[0335] Filed January 22, 2026 altitude m

[0336] The altitude of the camera above ground level, measured in meters,

[0337] sensor_height_mmm

[0338] The physical width of the camera sensor, measured in millimeters

[0339] focal_length_mm

[0340] The focal length of the camera lens, measured in millimeters.

[0341] image_width_px

[0342] The width of the image in pixels.

[0343] gsd_vertical_m

[0344] Ground Sample Distance in the vertical direction.

[0345] altitude_m

[0346] The altitude of the camera above ground level, measured in meters.

[0347] sensor_height_mm

[0348] The physical width of the camera sensor, measured in millimeters

[0349] image_height_px

[0350] The height of the image in pixels.

[0351] 6.3 Diagonal Attributes

[0352] For diagonal features (e.g., hips, valleys), the GSD is calculated as:

[0353] gsd_diagonal_m = (gsd_horizontal_m + gsd_vertical_m) / 2

[0354] This approach better approximates real-world distances for non-axis-aligned roof edges, gsd vertical m is used for calculating the lengths of vertical features in an image. In the context of roof measurements, elements such as eaves, ridges, and rakes can appear in any orientation.

[0355] Some features (such as ridges and eaves) may be aligned horizontally (in two dimensional x-y space)), while others may be aligned vertically in t 'O dimensional x-y space). For features aligned horizontally, the horizontal GSD is used for length calculation. For features aligned vertically, the vertical GSD is used. This distinction ensures accurate real-world measurements regardless of the orientation of the roof feature within the image.

[0356] 7. Final Length Computation

[0357] The final real-world length for each instance is computed as:

[0358] length_meters = pixel_length * applicable_gsd

[0359] length_feet = length_meters * 3.28084Docket No. 2954678-00004-W01

[0360] Filed January 22, 2026 Where applicable gsd is chosen based on the orientation of the instance.

[0361] Each roof attribute instance therefore has: - Independent geometry - Independent pixel length - Orientation-aware real-world measurement.

[0362] GSD is used only for computing the real-world lengths of individual roof features (such as eaves, ridges, and rakes). These lengths are derived by converting pixel distances to real- world distances using the appropriate horizontal or vertical GSD, depending on the feature’s orientation.

[0363] A scale factor is used to convert measurements from pixel space to real-world units when calculating the area of the entire roof

[0364] To compute the scale factor, we use known linear measurements rather than applying GSD duecby. the piocess is as lollov / s.

[0365] 1. Calculate the total pixel length

[0366] Sum the pixel lengths of all roof features (such as eaves, ridges, rakes, hips and valleys). This represents the total length measured in pixel units.

[0367] 2. Calculate the total real-world length

[0368] Convert each feature’s pixel length to real-world units (feet) using the appropriate GSD (horizontal or vertical), and then sum these values to obtain the total real-world length.

[0369] 3. Compute the scale factor

[0370] Divide the total real-world length by the total pixel length:

[0371] 4. Apply the scale factor to area calculation

[0372] When computing the roof area, the polygon area is first calculated in pixel units. This pixel-based area is then converted to real-world area by applying the scale factor consistently across the entire polygon.

[0373] This computation is further described below.

[0374] 8. Closed-Surface Area Measurement (Implementation-Level Explanation)

[0375] This section explains exactly how roof surface areas are computed in the backend.

[0376] 8.1 Line Normalization and Intersection Splitting

[0377] Before any area computation is attempted, all roof edge annotations are normalized and split at geometric intersections.Docket No. 2954678-00004-W01

[0378] Filed January 22, 2026 Why this step is critical: - Roof faces are bounded by multiple roof attributes (eaves, rakes, hips, ridges) - Long edges often have other edges terminating on them - If lines are not split at intersection points, polygon construction will be incorrect

[0379] What the function does: 1. Collects all unique endpoints from every annotation

[0380] If there are three annotations:

[0381] • eave 1 •••••> endpoints: (10, 20), (100, 20)

[0382] • eave_2 — * endpoints: (100, 20), (180, 20)

[0383] • ridge 1 endpoints: (50, 80), (150, 80)

[0384] The system initially collects'

[0385] • (10, 20), (100, 20), (100, 20), (180, 20), (50, 80), (150, 80)

[0386] After removing duplicates:

[0387] • (10, 20), (100, 20), (180, 20), (50, 80), (150, 80)

[0388] 2. Detects points that lie collinearly on another line segment 3. Splits a long line into smaller segments wherever another edge connects.

[0389] As shown in the Figure 11, the line 1110 (attaching 1102 to 1104) has one endpoint that lies on an eave (1100) and another endpoint that lies at a junction 1102

[0390] The system detects points that lie collinearly on an existing line segment, meaning it identifies points where another instance intersects or lies directly along the same straight path as the eave.

[0391] When such a point is detected, the system splits a longer line into smaller line segments at the point of intersection. In this example, the original long eave (line 1102) is divided into two separate eave segments at the location where line 1110 meets it

[0392] 4. Preserves real-world length proportionally to pixel length.

[0393] This guarantees a topologically consistent edge graph, which is mandatory for closed-loop detection.

[0394] 8.2 Graph Construction from Roof Edges

[0395] Once annotations are split, the system builds an undirected planar graph:

[0396] • Each unique point becomes a node

[0397] In this context, a node is intended to represent a junction where two or more-line instances (edges) meet.

[0398] To clarify:Docket No. 2954678-00004-W01

[0399] Filed January 22, 2026 The system does not treat every extracted po nt or endpoint as a node

[0400] A point becomes a node only when it represents a junction, meaning it is shared by two or more annotations or line segments.

[0401] Such a node is not unique to a single instance; instead, it is a shared connection point between multiple edges.

[0402] • Each line segment becomes an edge

[0403] • Adjacency is stored bidirectionally

[0404] When adjacency is stored bidirectionally, it means that each node explicitly records which edges (line instances) are connected to it, and each edge is also aware of the nodes at its endpoints.

[0405] During this phase: - Pixel-space endpoints are rounded for stability - Each edge stores its real-world length (already converted using GSD earlier in the pipeline). Rounding a pixel space endpoint simply means rounding the coordinate values up or down to the nearest whole number.

[0406] This graph is the structural foundation used to detect closed roof faces.

[0407] 8.3 Directed Edge Traversal for Face Detection

[0408] To detect closed roof surfaces, the algorithm performs directed edge traversal, inspired by planar graph face-walking techniques.

[0409] Implementation highlights: - For each node, neighboring nodes are sorted by polar angle -Each edge is treated as a directed edge (u — > v) - Traversal always selects the next edge by taking the right-most turn

[0410] A neighbouring node is any node that i one edge (one hop) away.

[0411] For each node, its directly connected neighbours are sorted by polar angle. Each undirected edge is treated as two directed edges in the following manner:

[0412] For an edge between u and v, create: u-->v and v-»u

[0413] Now', when traversing:

[0414] u w: belongs to one face

[0415] v—*u: belongs to the adjacent face on the other side

[0416] The algorithm then performs directed edge traversal, starting from each unused directed edge, and at each step selects the next edge by making a right-most turn based on the angular ordering. This process walks the boundary of a face until a closed loop is formed. The traversal is repeated globally until all directed edges are exhausted.Docket No. 2954678-00004-W01

[0417] Filed January 22, 2026 This approach: - Walks the perimeter of a face - Terminates when the traversal returns to the starting edge

[0418] Invalid or open paths are discarded automatically.

[0419] 8.4 Polygon Formation in Pixel Space

[0420] If a traversal forms a valid loop: - The ordered sequence of points is treated as a polygon. A minimum pixel-area threshold is applied to remove noise [add a sentence to clarify]

[0421] At this stage, the roof surface exists purely in pixel coordinates, independent of scale or units.

[0422] 8.5 Pixel Area Computation (Shoelace Formula)

[0423] For each valid polygon, the area is computed using the shoelace formula:

[0424] Area_px = 1 / 2 • |L(x_i • y_{i+l }_x_{i+l } • y_i)|

[0425] 8.6 Conversion from Pixel Area to Real-World Area

[0426] Instead of directly applying GSD, the implementation derives a scale factor from perimeter measurements:

[0427] 1. Pixel perimeter is computed during face traversal

[0428] 2. Real-world perimeter is computed by summing edge real lengths

[0429] 3. Scale factor is calculated as:

[0430] scale = real perimeter / pixel_perimeter

[0431] 4. Final real-world area is computed as:

[0432] Area_real = Area_px x (scale2)

[0433] This approach accounts for mixed edge orientations - avoids compounding GSD errors -maintains consistency with length calculations

[0434] The output unit is square feet, matching downstream reporting requirements.

[0435] 8.7 Duplicate Face Detection and Area Normalization

[0436] Due to bidirectional traversal, the same physical roof face may be detected multiple times.Docket No. 2954678-00004-W01

[0437] Filed January 22, 2026 To resolve this: - Faces are grouped by a signature based on side count and sorted edge lengths - Areas for matching faces are averaged - All duplicates are normalized to the same final area value

[0438] This ensures each physical roof face is counted exactly once.

[0439] The systems and method described for detecting roof attributes from aerial imagery provide the following advantages:

[0440] Accurate per-instance roof attribute detection

[0441] Reliable and explainable length calculations

[0442] Elimination of heuristic GSD tuning

[0443] Improved robustness across different roof geometries

[0444] Added area calculation

[0445] Cleaner integration with downstream reporting and costing modules

Claims

Docket No. 2954678-00004-W01Filed January 22, 2026 CLAIMS1. A method comprising,receiving an aerial image of a roof structure;receiving image information of the aerial image, wherein the image information includes pixel length of the image, pixel height of the image, and altitude of the sensor capturing the image;using a trained model to analyse the aerial image and predict linear features of the roof structure, wherein the linear features comprise eaves, ridges, hips, valleys, and rakes;converting each linear feature into a geometric representation, wherein each geometric representation comprises an ordered list of two dimensional pixel coordinates in cartesian space;computing sensor parameters using the image information, wherein the sensor parameters comprise sensor width and sensor height;converting the geometric representations of the linear features into physical length dimensions using the sensor parameters and the image information;computing surface area of the roof structure using the geometric representations of the linear features and their corresponding physical length dimensions.

2. The method of claim 1, wherein the converting the geometric representations of the linear features comprises determining pixel length of each geometric representation.

3. The method of claim 2, wherein the converting the geometric representations of the linear features comprises determining Euclidean distance between consecutive points of each geometric representation.

4. The method of claim 3, wherein the converting the geometric representations of the linear features comprises determining ground sample distance parameters for the linear features of the roof structure.

5. The method of claim 4, wherein the converting the geometric representations of the linear features comprises multiplying pixel length of each geometric representation by its respective ground sample distance parameter.Docket No. 2954678-00004-W01Filed January 22, 20266. The method of claim 2, wherein the computing the surface area comprises representing the geometric representations as an undirected planar graph.

7. The method of claim 6, wherein the computing the surface area comprises traversing the undirected planar graph to identify discrete polygonal components of the undirected planar graph.

8. The method of claim 7, wherein the computing the surface area comprises determining pixel perimeter of the graph by summing the pixel length of the geometric representations.

9. The method of claim 8, wherein the computing the surface area comprises determining pixel area of each discrete polygonal component.

10. The method of claim 9, wherein the computing the surface area comprises determining total pixel area of the undirected planar graph by summing the pixel area of the discrete polygonal components.

11. The method of claim 10, wherein the computing the surface area comprises determining total physical perimeter of the undirected planar graph.

12. The method of claim 11, wherein the determining the total physical perimeter of the undirected planar graph comprises summing the physical length dimensions of the geometric representations.

13. The method of claim 12, wherein the computing the surface area comprises generating a ratio parameter equal to the physical length perimeter of the undirected planar graph divided by the pixel perimeter of the undirected planar graph.

14. The method of claim 13, wherein the computing the surface area comprises multiplying total pixel area of the undirected planar graph by the ratio parameter.

Images