Vehicle and tire inspection with 3d-equipped user devices

Abstract

Embodiments of the disclosure relate to a method for inspecting a vehicle component. In the method, three-dimensional spatial data of the vehicle component is captured using a depth-sensing subsystem of a user device. Two-dimensional image data of the vehicle component is also captured using an imaging subsystem of the user device. A fused representation of the vehicle component is generated by spatially aligning the three-dimensional spatial data with the two-dimensional image data. Geometric features are extracted from the three-dimensional spatial data and visual features from the two-dimensional image data. The extracted geometric and visual features are provided as inputs to one or more trained machine-learning models. Using the trained machine-learning models, a physical measurement or condition metric of the vehicle component is derived based on the fused representation. An inspection result is generated indicating whether the physical measurement or condition metric satisfies a regulatory or maintenance criterion.

Description

VEHICLE AND TIRE INSPECTION WITH 3D-EQUIPPED USER DEVICESCROSS-REFERENCE TO RELATED PATENT APPLICATIONS

[0001] This patent application claims the benefit of U. S. Provisional Patent Application No. 63 / 755,261, filed February 7, 2025, the entire teachings and disclosure of which are incorporated herein by reference thereto.FIELD OF THE INVENTION

[0002] The present invention relates generally to a system and method for inspecting vehicle tires and components and, in particular, to a system and method of using a user device equipped with 3D sensors, including LiDAR, depth cameras, and stereo vision systems to scan and inspect vehicle tires and components.BACKGROUND OF THE INVENTION

[0003] The field of vehicle maintenance and inspection has seen significant advancements, with various tools and systems designed to monitor vehicle health and ensure compliance with safety regulations. However, existing technologies often lack the ability to perform comprehensive, real-time scanning, detect DOT out-of-service criteria, and integrate seamlessly into modem fleet management systems. Below is an overview of the art known to Applicant, referencing relevant patents and publications, and their limitations compared to embodiments of the present disclosure.

[0004] In the field of manual inspections, manual methods remain the most traditional approach for vehicle inspections, including checking tire tread depth, brake wear, and other components using physical tools. These inspections rely on devices like tread depth gauges and torque wrenches, which are labor-intensive, prone to human error, and inefficient for large-scale fleet operations. U. S. Patent No. 3,170,243A, for example, describes a mechanical tread-depth gauge that provides basic measurements but lacks automation or advanced data integration.

[0005] Tire Pressure Monitoring Systems (TPMS) are widely used to monitor tire pressure and temperature in real time, ensuring vehicle safety. While effective for pressure- 160065464related issues, TPMS cannot detect critical problems such as tread wear, sidewall damage, or DOT out-of-service criteria. For example, U. S. Patent No. 7,225,973B2 describes a TPMS for monitoring pressure and temperature but does not address tread wear or other physical defects.

[0006] Telematics-Based Fleet Management Systems provide fleet operators with insights into vehicle location, engine performance, and driver behavior. However, they do generally not include capabilities for real-time scanning of physical components such as tires, brakes, and axles. U. S. Patent No. 8,190,303B2. for example, covers a telematics system for vehicle diagnostics and fleet management but lacks integration with scanning hardware or DOT compliance checks.

[0007] Portable Tread Scanners are handheld devices, such as laser-based tread depth scanners, that offer more precise measurements of tire wear compared to manual methods. However, these devices are limited to single-use applications, require significant user intervention, and lack backend integration for fleet-wide analytics or regulatory’ compliance. U. S. Patent No. 6,519,500B2 describes a laser-based tread depth measurement tool but does not address real-time data processing or multi-component scanning.

[0008] Augmented Reality (AR) has been explored as a tool for vehicle maintenance. For example, U. S. Publication No. 2019 / 0278992A1 describes an AR system that overlays visual guidance to assist users with vehicle maintenance tasks. However, it lacks real-time scanning, multi-component analysis, and integration with DOT compliance checks. U. S. Publication No. 2020 / 0026257A1 expands on AR for vehicle maintenance but is primarily focused on static overlays rather than dynamic defect detection or regulatory assessments.

[0009] Stationary inspection systems may include drive-over and in-ground systems used in some commercial applications to measure tire tread depth and alignment. While accurate, these systems are expensive, stationary, and limited to specific use cases like tire inspections, making them unsuitable for fleets requiring frequent or distributed inspections. U. S. Patent No. 9,123,842B2 describes an in-ground system for measuring tread depth but does not extend to scanning other vehicle components or enabling mobile inspections. U. S. Patent No.10,063,837B2 discloses a stationary automated inspection system but is limited to tire wear and alignment, lacking broader functionality for multi-component scanning or DOT compliance.260065464

[0010] Predictive maintenance using Al involves Al-driven solutions that analyze historical data to predict maintenance needs. While useful, these systems often rely on aggregated data and overlook real-time scanning of physical components like tires, brakes, or lights. For example, U. S. Patent No. 10,295,333B2 describes a system for predictive maintenance but lacks capabilities for real-time scanning or regulatory compliance detection.

[0011] Vehicle Component Inspection Systems, such as disclosed in U. S. Patent No.10,247,641B2, provide a method for inspecting vehicle components using sensors to identify wear and abnormalities. While innovative, it focuses on specific components and does not leverage extended reality (XR), real-time user guidance, or integration with DOT compliance checks.

[0012] Enhanced tire tread measurement systems include laser-based and imageprocessing techniques for assessing tread wear. These systems provide better precision but are limited to tire-specific applications and often require separate, stationary setups. Two examples include U. S. Patent No. 7,578,180B2, which describes a compact device for tire tread depth measurement but lacks integration with fleet systems, and U. S. Publication No.2017 / 0176176A1, which discusses a handheld laser measurement device but does not address broader fleet applications or real-time regulatory checks.

[0013] Based on the foregoing discussion, the known art includes several significant limitations. First, most solutions address specific aspects of vehicle health, such as tire pressure or historical data analysis, without providing a comprehensive view of vehicle components. Second, many of the known systems are immobile, and such stationary systems are impractical for fleets requiring frequent inspections. However, known handheld tools are limited in scope. Third, existing systems fail to proactively identify DOT out-of-service criteria for tires and other components. Fourth, AR-based solutions lack real-time scanning capabilities, and manual methods offer no dynamic feedback. Fifth, data collected by the existing systems is isolated, and such siloed data prevents seamless fleet-wide integration and analysis.BRIEF SUMMARY OF THE INVENTION

[0014] Embodiments of the present disclosure relate to systems and methods for inspecting, scanning, analyzing, and maintaining vehicle components, including but not 360065464limited to tires, brakes, axles, lights, and other critical parts of a vehicle. Specifically, the invention employs advanced mobile and extended reality (XR) devices equipped with one or more 3D sensors, such as light detection and ranging (LiDAR) sensors, as well as cameras, including depth cameras and dual cameras in stereo vision, to capture, process, and analyze data related to vehicle components. Embodiments of the disclosed system enable the detection of Department of Transportation (DOT) out-of-service criteria, wear patterns, and other maintenance-critical issues.

[0015] Embodiments of the present disclosure further encompass applications in fleet management, predictive maintenance, and regulatory compliance. By integrating artificial intelligence, point cloud processing, image processing, and real-time feedback mechanisms, the invention provides actionable insights to enhance vehicle safety, reduce operational costs, and ensure compliance with DOT and other safety standards. The invention is particularly relevant to industries relying on large vehicle fleets, such as transportation, logistics, and commercial operations, where proactive maintenance and regulatory compliance are critical.

[0016] Embodiments of the present disclosure address these foregoing limitations by introducing a scalable, comprehensive, and integrated solution for vehicle maintenance and regulatory compliance. In one or more embodiments, the presently disclosed system incorporates extended reality (XR), which includes augmented reality (AR), virtual reality (VR), and / or mixed reality (MR) technologies, for real-time scanning and analysis of tires, brakes, axles, lights, and other components. Further, in one or more embodiments, the presently disclosed system provides DOT compliance detection by identifying out-of-service criteria and providing actionable recommendations to ensure compliance.

[0017] One or more embodiments of the presently disclosed system may incorporate multi-sensory user guidance. For example, the system may offer video tutorials, animations, auditory alerts, and haptic feedback during inspections and repairs.

[0018] One or more embodiments of the presently disclosed system involve cloud-based integration in which data is centralized and exposed via application programming interfaces (APIs) for telematics, maintenance scheduling, and regulatory compliance reporting.60065464

[0019] One or more embodiments of the presently disclosed system provide mobile scalability utilizing portable devices with LiDAR, depth cameras, and / or 3D sensors for efficient, on-the-go inspections.

[0020] Using the presently disclosed system, fleet-wide insights may be delivered via dashboards, reports, and real-time or near real-time alerts, thereby improving operational efficiency and safety.

[0021] Embodiments of the present disclosure relate to systems and methods for inspecting, scanning, and analyzing vehicle maintenance items, such as measuring tire tread depth and wear, particularly for, but not limited to, commercial vehicles, using mobile and extended reality devices equipped with 3D scanning hardware, such as LiDAR, depth cameras, or dual cameras in stereo vision. The devices may be equipped with one or more cameras to further enhance vehicle maintenance data analysis and capture video inspection evidence. Measurements are computed based on the scanned 3D point cloud data as well as imagery data collected via the cameras during the scan using a combination of one or more point cloud processing algorithms, artificial intelligence, image processing algorithms, and any other useful advanced algorithms either directly on the device or on a backend server. The 3D data and imagery data collected from the device during the scans are used for calculations including, but not limited to, tread depth measurement, tire wear pattern detection, predictive maintenance recommendations, wear metric calculations, maintenance reports tailored for users, detecting DOT out-of-service items, and so on.

[0022] The scan results are analyzed and monitored to give tailored alerts to fleet users and owners that can use this information to accurately maintain and optimize their vehicles. Reports are generated based on the collected 3D point cloud and imagery data in order to provide sustained insights for fleets that want to keep on top of their tire and vehicle maintenance program. Collected vehicle maintenance data, including 3D point cloud and image data, is also shared with third parties upon permitted request to promote continued use and integration of the application. In one or more embodiments, app users are provided with digital rewards, such as cryptocurrency, for scanning and labeling vehicle maintenance items, such as tires, in order to build machine learning datasets for data processing pipelines. Embodiments of the present disclosure address the need for accurate, efficient, and automated tire tread analysis to enhance vehicle safety and reduce maintenance costs. The system560065464uniquely distinguishes itself as a novel solution for vehicle inspections and maintenance with advanced 3D scanning technology, cutting edge Al for data analysis, fast on-device processing capabilities, on-demand DOT compliance checking, predictive maintenance and compliance alerts, third-party data sharing, and a highly user-friendly interface.

[0023] Other aspects, objectives, and advantages of the invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings incorporated in and forming a part of the specification illustrate several aspects of the present invention and, together with the description, serve to explain the principles of the invention. In the drawings:

[0025] FIG. 1 shows a schematic overview- of the system, according to one or more exemplary embodiments of the present disclosure;

[0026] FIG. 2 show s a user device, such as a tablet, held by a user scanning a tractortrailer tire using a LiDAR sensor native to the user device, according to one or more exemplary embodiments of the present disclosure;

[0027] FIG. 3 shows a user wearing an XR headset and scanning a tractor-trailer using the XR headset's native LiDAR sensors while interacting with the scanned representation in the headset, according to one or more exemplary embodiments of the present disclosure;

[0028] FIG. 4A shows the vehicle component scan data collection, preprocessing, and storage flow, according to one or more exemplary embodiments of the present disclosure;

[0029] FIG. 4B shows the vehicle component scan full data flow, according to one or more exemplary embodiments of the present disclosure;

[0030] FIGS. 5A-5N depict app screens for conducting a vehicle inspection involving measurement of tire tread depth and wear properties;

[0031] FIG. 6 depicts the point cloud data 230 and imagery data 240 as captured during a scan;660065464

[0032] FIG. 7A shows the first half of the data processing pipeline for scanned point cloud and imagery data, which may be executed on the device or in the cloud, according to one or more exemplary embodiments of the present disclosure;

[0033] FIG. 7B shows the second half of the data processing pipeline for scanned point cloud and imagery data, which may also be executed on the device or in the cloud, according to one or more exemplary embodiments of the present disclosure;

[0034] FIG. 8 A shows a diagram of how tire tread depth is measured from a rear-facing view of a tractor-trailer tire tread along its surface, according to one or more exemplary embodiments of the present disclosure;

[0035] FIG. 8B shows a tractor trailer tire with arrows pointing to sample tread depth measurement points, according to one or more exemplary embodiments of the present disclosure;

[0036] FIG. 9 shows a tire wear analysis pipeline that uses the current and existing scan data to analyze the tires and provide wear analysis results, reporting, and maintenance suggestions, according to one or more exemplary embodiments of the present disclosure;

[0037] FIG. 10 shows a fleet maintenance dashboard displaying tire health statistics, analysis, and maintenance alerts, according to one or more exemplary embodiments of the present disclosure;

[0038] FIG. 11 shows the system integrations with third-party systems and APIs, according to one or more exemplary embodiments of the present disclosure;

[0039] FIG. 12 depicts a pipeline for data preparation and model training, according to one or more embodiments of the present disclosure;

[0040] FIG. 13 is a flow diagram of a multi-frame data capture and point cloud reconstruction, according to one or more embodiments of the present disclosure;

[0041] FIG. 14 is a data pipeline for point-cloud filtering, tire surface isolation, and canonical preparation, according to one or more embodiments of the present disclosure;760065464

[0042] FIG. 15 depicts an architecture for a multi-modal machine learning model, according to one or more embodiments of the present disclosure;

[0043] FIG. 16 depicts an end-to-end pipeline for backend processing, according to one or more embodiments of the present disclosure;

[0044] FIG. 17 depicts a flow diagram illustrating a comparison between a tire tread scan and a tire sidewall scan, according to one or more embodiments of the present disclosure;

[0045] FIG. 18 depicts a diagram for data structures and dataset format, according to one or more embodiments of the present disclosure; and

[0046] FIG. 19 depicts a pipeline for tire tread measurement using LiDAR and a machine learning implementation, according to one or more embodiments of the present disclosure.

[0047] While the invention will be described in connection with certain preferred embodiments, there is no intent to limit it to those embodiments. On the contrary, the intent is to cover all alternatives, modifications and equivalents as included within the spirit and scope of the invention as defined by the appended claims.DETAILED DESCRIPTION OF THE INVENTION

[0048] Referring generally to the figures, various embodiments of a system and method for performing real-time scanning, analysis, and predictive maintenance of vehicles, especially tractor-trailers, are provided. As will be discussed more fully below, the disclosed system and method are expected to enhance safety and regulatory compliance by measuring tire tread depth, detecting DOT out-of-service criteria, predicting failures, and providing maintenance recommendations using Al-driven models, point cloud processing, and image analysis.

[0049] In various embodiments, vehicle inspections can be conducted on-device or via a cloud-based backend, leveraging lightweight mobile implementations for real-time results. The system may be used to identify wear patterns, alignment issues, and potential maintenance needs, providing actionable insights through an intuitive application. Additionally, in embodiments, the application includes video tutorials, animations, and multi-860065464sensory feedback to guide users during scans. A fleet management web portal offers detailed reports, compliance alerts, predictive maintenance schedules, and cost-savings analysis.

[0050] Advantageously, the application may provide seamless integration with third-party systems and APIs, including fleet management, transportation management, telematics, maintenance platforms, electronic driver vehicle inspection reports (eDVIR), electronic logging devices (ELD), Department of Transportation (DOT), Federal Motor Carrier Safety Administration (FMCSA), and tire manufacturers, which provide efficient data sharing and fleet-wide optimization. Additionally, the app may be used to administer an incentive program, rewarding users with cryptocurrency or point-based benefits for high-quality scans and maintenance compliance. In this way, the presently disclosed system and method may allow for improved operational efficiency, safety, and cost-effectiveness in commercial fleet maintenance.

[0051] These and other aspects and advantages of the disclosed vehicle inspection system and method will be discussed more fully below in relation to the embodiments provided herein and shown in the figures. These embodiments are provided by way of illustration and not limitation.

[0052] Referring generally to FIGS. 1-11, various embodiments of the present disclosure relate to a system for vehicle and tire inspection with 3D-equipped user devices, such as mobile and extended reality (XR) devices.

[0053] In general, embodiments of the presently disclosed system 100 may involve five elements. With reference to FIGS. 1 -4B, a first element is a user device 105, such as a mobile device 120 or XR device 180. As will be explained below, the user device 105 captures 3D point cloud data 230 and imagery data 240 for tires and vehicle components using 3D-capable sensors, such as LiDAR 210, depth cameras 220, or dual cameras 220 in stereo vision, as well as 2D-capable sensors, such as traditional 2D cameras 220. The use of such 2D and 3D sensors allows for the creation of high-resolution 3D models of the vehicle 125, including various components, in particular tire treads. The data 230, 240 is stored on device storage 260 of the user device 105. In one or more embodiments, the user device 105 performs realtime processing 410 to, e.g., measure tread depth, detect DOT out-of-service criteria, provide corrective guidance, etc. In conjunction with user input data 200, the 3D point cloud data 230 and imagery data 240 are uploaded 270 to a backend 170 using a communication network. In 960065464one or more embodiments, the communication network is a wireless internet connection over Wi-Fi or cellular, including 3G, 4G, 5G, and so on.

[0054] A second element of the system 100 is the user application (or "app") 106. A user-friendly app 106 provides functionality for scanning, preparing, analyzing, reviewing, labeling, and managing data collected by the user device 105. The app 106 may feature realtime analysis and feedback, DOT compliance checks, user rewards, and integration with third-party systems. The app 106 is executed on the user device 105. In one or more embodiments, the app 106 is a proprietary fleet management app. That is, the app 106 may be issued to personnel involved in maintenance or operation of a company's vehicle fleet. In one or more other embodiments, the app 106 is an app designed for individual users. In still one or more other embodiments, the presently disclosed system operates as a module of a third-party app via a software development kit (SDK), thereby enabling third parties to integrate the presently disclosed scanning technology into their inspection apps.

[0055] A third element of the system 100 is a plurality of data processing pipelines. As will be discussed below, these data processing pipelines utilize Al models, point cloud processing, and image processing algorithms to compute wear metrics, predict failures, and derive actionable insights.

[0056] A fourth element of the system 100 is a backend system 170, in particular a cloudbased backend. As used herein, the term "backend system" refers to the location where data from one or more users 110 and / or fleet management personnel 150 is collected and / or processed. The backend system 170 offers secure cloud storage 171, high-performance computing, and web services to process, analyze, and synchronize fleet-wide scan and maintenance data. In this way, the backend system 170 provides high-performance computing to execute data processing pipelines, including real-time and batch DOT compliance checks. Al-driven data analysis, and predictive maintenance insights. Additionally, the backend system 170 provides scalable and secure cloud storage for retaining scanned data, tire tread measurements, historical records, and derived insights, with encryption and access control for data security. Still further, backend system 170 provides high-speed, secure networking to enable low-latency large data transmission, rapid API responses, and real-time data synchronization across a distributed fleet operation. The backend system 170 may also provide comprehensive web services for hosting RESTful APIs, web applications, and1060065464databases, facilitating seamless communication between apps, devices, and cloud systems. Batch and stream processing capabilities may be provided to handle fleet- wide scans, trend analysis, and anomaly detection based on historical data. Additionally, APIs can be seamlessly integrated with third-party fleet management systems, telematics platforms, and compliance reporting tools to enhance operational efficiency.

[0057] A fifth element of the system 100 is a web platform 161 accessible by fleet management devices 160. Fleet management devices 160 are used by fleet management personnel 150, also referred to as fleet users 150, to access the web platform 161. The web platform 161 provides dashboards, alerts, and reports for fleet health and maintenance. In one or more embodiments, the web platform 161 integrates with various systems and APIs 300, including such systems and APIs as fleet management 300A, transportation management 300B. telematics 300D. maintenance 300C, electronic logging device (ELD) 300E, electronic driver vehicle inspection reports (eDVIR) 300F, and safety 300G, as shown in FIG. 11, to derive the best insights for the fleets and share data across platforms.

[0058] FIG. 1 shows a schematic overview of the system 100 according to one or more embodiments of the present disclosure. As illustrated in FIG. 1, the system 100 includes at least one user device 105, such as a mobile device 120 or an XR device 180. The user device 105 executes an app 106. For example, if the user device 105 is a mobile device 120 (e.g., iPad Pro or iPhone Pro), the mobile device 120 may execute a mobile app 121, and if the user device 105 is an XR device (e.g., Apple Vision Pro or Meta Quest 3), the XR device 180 may execute an XR app 181. Other user device types are also possible, and the examples provided are merely illustrative. In embodiments in which multiple user devices 105 are utilized, a single user 110 may use the user devices 105 simultaneously. In other embodiments, the mobile device 120 and the XR device 180 are not used simultaneously by the same user 110; instead, the mobile device 120 and the XR device 180 may be used at different times by the same user 110 or by multiple users 110 within the system 100.

[0059] In one or more embodiments, a user 110, such as a driver or maintenance technician, of a vehicle 125 utilizes the app 106 to conduct an inspection of the vehicle 125. In the embodiment depicted in the figures, the vehicle 125 is a tractor 130 that may pull one or more trailers 140. However, in one or more other embodiments, the app 106 may be utilized with other vehicles, such as cars, trucks, vans, light trucks, and medium trucks, amongst other1160065464possibilities. The user 110 may utilize the mobile app 121, the XR app 181, or both to perform vehicle inspections. For example, the user 110 may use a particular app 106 (e.g., mobile app 121 or XR app 181) for certain portions of an inspection of the vehicle and another app 106 for other portions of the inspection. In another example, the user 110 may use one particular app 106 (e.g., mobile app 121 or XR app 181) for inspections of particular vehicles 125 or parts of a vehicle 125 and another app 106 for inspections of other particular vehicles 125 or other parts of the vehicle 125.

[0060] In one or more embodiments, the vehicle 125 is, in particular, a commercial vehicle that is part of a fleet, for example. In such embodiments, fleet personnel 150, such as a fleet manager, maintenance team member, or safety team member (amongst other possibilities), using the fleet device 160 may be provided with information generated by the user device 105 through the app 106. The app 106 is configured to communicate over a network with the backend system 170. In this way, information generated or input through the app 106 by the user device 105 can be utilized by fleet personnel 150 using the fleet management device 160 accessing a web app 161 (which may also be referred to herein as "web portal," "web platform," and "website"). In the embodiment illustrated in FIG. 1, the backend system 170 is a cloud-based platform. Collected data may be stored locally for each user 110 on the device 105 and collectively for a plurality of users 110 on the cloud-based platform. Further, the data may be processed either on device 105 or through the backend system 170 or using a combination of the two.

[0061] As will be explained in more detail below, embodiments of the presently disclosed system and method provide superior detail, accuracy, and speed for conducting vehicle inspections compared to traditional photogrammetry-based approaches. Using the user devices 105, users 110 can initiate scans, visualize results in real-time, and generate comprehensive reports through the intuitive app 106. Embodiments of the app 106 prioritize accessibility and ease-of-use, even for users 110 without technical expertise.

[0062] FIG. 2 depicts a user 110 scanning a tire 600 using a user device 105, in particular a mobile device 120. In one or more embodiments, the scanned tire data points display as an overlay on the video on the screen of the device 120 while the user 110 scans points on a given tire. Using three-dimensional point cloud data, the mobile device 120 can overlay the scanned points on the video of the tire as it is captured in real-time. In one or more1260065464embodiments, the entire tire point cloud data 230 with imagery data 240 is reconstructed on the device 120 once the scan is completed. In one or more embodiments, the user 110 can visualize the tire scan data 230, 240 as well as the tire reconstruction on the device 120 in a 3D viewer of the mobile device 120. In one or more such embodiments, the user 110 can rotate and zoom in on the 3D tire, for example on the treads or sidewall, to inspect the tire at a higher level of detail than a typical manual inspection based on the raw scan data collected.

[0063] FIG. 3 depicts a user 110 scanning a vehicle 125, in particular having a tractor 130 and a trailer 140. using another user device 105, in particular an XR headset 180. The 3D representation of the vehicle 125, including tractor 130 and trailer 140, is shown in the headsup display (HUD) of the headset worn by the user 110. The user 110 is able to visualize the scanned data points of the vehicle 125 overlaid on the image data 240 captured by the XR device 180 as the XR device 180 is worn by the user 110 so that the 3D vehicle point cloud data 230 appears to be registered with the physical vehicle 125. The scanned representation can be zoomed and altered with various gestures as is known in the art in order to get a highly detailed view of the vehicle 125 or components of the vehicle 125 for maintenance and safety analysis.

[0064] To initiate a vehicle inspection, the user 110 opens the app 106 on the user device 105. In one or more embodiments, the user 110 creates a user account and associates a company or fleet with the account. In one or more embodiments, fleet personnel 150 acting as account administrators may create user accounts in the system 100, granting specific permissions and roles to users 110 within the app 106. The fleet personnel 150 input information into the system 100 about the fleet, including but not limited to, company information (such as vehicle information; driver information; and maintenance crew¬ information), provider information (such as ELD, telematics, TMS. FMS, maintenance systems, and eDVIR); relevant third-party API 300 connection details including credentials for providers; and maintenance schedules. Fleet information may be auto-filled by the system 100 using third-party system and API integrations 300 depending on the scenario. Advantageously, the system 100 ensures fleet data accuracy and allows for future updates.

[0065] To add a vehicle 125 to the system 100, a user 110 or fleet management personnel 150 creates the vehicle 125 via the app 106 or via the web platform 161. When creating the vehicle 125, several items of information about the vehicle 125 may be entered, such as1360065464vehicle type, year, make, model. VIN. and mileage. This information may automatically be pulled by the system 100 using third party systems and APIs 300. The user 110 or fleet personnel 150 then saves this information in the app 106. The vehicle 125 is created in the system 100 and the information is saved to device storage 260 of the user device 105 and to cloud storage 171 of the backend system 170. Once the vehicle 125 exists in the system 100, the user 110 is then prompted to configure the positions of the tires 600 on the vehicle 125 in the system 100 for the first time using a tire position function before scanning can occur.

[0066] In the tire position function, the user 110 or fleet personnel 150 can set the configuration of tires 600 for the vehicle 125 depending on whether the vehicle 125 is a tractor 130, a trailer 140, both, or another vehicle type. The user 110 or fleet personnel 150 is shown a diagram of the layout of the tires 600 of the vehicle 125 based on the vehicle information gathered from the user 110 or fleet personnel 150 including the type of the vehicle 125, axle configuration, number of tires 600, positions of the tires 600, etc. For each tire 600 in the diagram, the user 110 or fleet personnel 150 selects the tire 600 and is prompted for mandatory tire 600 information such as year, make, model, tire size, and serial number. For each tire 600, the system 100 provides the functionality to recognize one or more pieces of tire information, such as tire serial number, make, model, or size, from an image of the tire sidewall 650, e.g., using optical character recognition (OCR) from such tire information visible in the image. The user 110 may take a picture of the tire sidewall 650 with the tire information visible using the device 105, or the fleet personnel 150 may upload a similar picture of the tire sidewall 650 with the tire information for the system 100 to recognize. In such embodiments, the system 100 detects and recognizes the tire information from the provided picture of the tire sidewall 650 using one or more Al models and algorithms as listed below. The user 110 or fleet personnel 150 reviews the recognized tire information and can correct any incorrect information before proceeding.

[0067] In one or more embodiments, the app 106 may prompt the user 110 to calibrate the sensors of the user device 105, including the 3D scanner 210 and cameras 220. In such embodiments, calibration steps may include moving the user device 105 through specified motions to align the 3D sensors 210. to adjust the camera 220 settings for optimal focus and lighting conditions, and to verify calibration through a test scan. The calibration steps may be accompanied by visual and audio feedback provided by the app 106. For example, the app1460065464106 may provide short tutorial clips demonstrating the calibration process to ensure proper setup.

[0068] Once the tire information (serial number, make, model, and / or size) is recognized or provided, the system 100 decodes the tire information to derive additional manufacturing information about the tire 600 such as, but not limited to, ply rating, load range, maximum load, maximum inflation pressure, and position. The system 100 adds any additional manufacturing information that can be found to the tire 600 profde. Once the mandatory tire 600 information and positions are input by the user 110 or fleet personnel 150, the system 100 associates the identification information of the vehicle 125 provided by the user 110 or fleet personnel 150 with the tire 600 information and positions on the vehicle 125. Once configured, the vehicle 125 and tire 600 configuration data, including tire positions, are stored on device storage 260 and uploaded to cloud storage 171 for system-wide 100 reference.

[0069] Upon initially logging in or registering with the app 106, the user 110, preferably a maintenance technician, uses the app 106 to scan all the vehicles 125 for the fleet into the system 100 including tire 600 and vehicle 125 component scans. Maintenance technicians 110 typically pay more attention to detail and are aware of lower-level details related to the system 100 technologies and capabilities. Maintenance technicians 110 take detailed scans and provide precise labels for the scanned tire data, including scanned tire tread depth measurements entered via the app 106. The high-quality data scans and precise depth measurement labels are used to train accurate machine learning models in order to measure tire tread depth in an accurate and automated way via the data processing pipeline as discussed more fully below in relation to FIGS. 7A and 7B. The initial scan dataset for all the vehicles 125 sets a baseline for the data processing pipelines. Eventually, once enough data is collected for each vehicle 125 by the maintenance technicians 110, the drivers 110 can then use the app 106 to scan vehicle 125 components and complete pre-trip and post-trip eDVIR inspections, for example.

[0070] After this initial configuration setup, the vehicle 125 and tire 600 configurations must be updated by the maintenance team (users 110 and / or fleet personnel 150) either via the app 106 or via the web platform 161. Users 110 or fleet personnel 150 use the Tire Position function of the app 106 or web platform 161 to update and maintain the tire 600 positions on a selected vehicle 125 whenever tires 600 are rotated or replaced. In one or more1560065464embodiments, the Tire Position function of the system 100 also has a Tire Rotation mode, where the app 106 guides the user 110 through a customized tire 600 rotation sequence for the vehicle 125. The rotation sequence can be chosen by the user 110 or automatically determined by the app 106. The automatic tire 600 rotation sequence is generated based on Al analysis results produced by the data processing pipelines using the vehicle’s data as discussed below in relation to FIGS. 7A and 7B. For example, the system 100 may recommend a tire 600 rotation schedule that occurs every 3 / 32 inches of wear on the tires 600 but is customizable by the app users 110 and fleet personnel 150. The app 106 guides the users 110 through the tire 600 rotations in sequence, step by step, to update the positions and tire information efficiently. Desirably, users 110 or fleet personnel 150 keep the vehicle 125 and tire 600 information up to date to maintain system 100 accuracy and effectiveness.

[0071] After initializing the app 106 with user and fleet information, the app 106 can be used to carry out inspections of vehicle components. FIGS. 5A-5N depict app screens that walk the user 110 through a scan of vehicle components, such as tire treads. Referring to FIG.5A, a first app screen 801 encountered by the user 110 may be a login screen in which the user 110 is prompted to provide credentials, such as a user ID 820 (e.g., username or email address) and a password 821. As is known in the art, the credentials may be provided using biometric authentication, including facial recognition or fingerprint identification, a personal identification number (PIN), or via a passkey.

[0072] Thereafter, the user may be directed to a vehicle selection screen 802 as shown in FIG. 5B. On this screen 802, the user may select from a particular vehicle 125 or a part of the vehicle 125 for which the scan is being made. In the embodiment shown in FIG. 5B, the user 110 selects between a first button 822 to select a tractor 130 or a second button 823 to select a trailer 140. In an example embodiment in which the first button 822 is selected, the app 106 proceeds to a selection screen 803. On the selection screen 803, the user 110 is prompted to select or identify the particular tractor 130 being scanned. For example, the selection screen 803 may include a tractor ID or VIN prompt 824 in which the user enters the tractor ID or VIN (e.g., as shown in FIG. 5D), or the user 110 may be provided a list or dropdown menu of stored tractor IDs from which to select. Further, in one or more embodiments, the tractor ID or VIN is auto-detected using ELD 300E or telematics 300D API integrations based on the login information of the user 110. If the vehicle 125 entered or detected is not associated with the system 100 yet, then the vehicle 125 must first be created in the system 100 as 1660065464discussed above. In the embodiment shown in FIGS. 5C and 5D. the user clicks a continue button 825 after the desired tractor 130 is identified for scanning.

[0073] Because vehicles 125 include multiple tires and other components, especially tractors 130 and trailers 140. the app 106 includes a component selection screen 804. In the embodiment depicted in FIG. 5E, the app 106 is configured for scanning of tires in particular, and thus, the component selection screen 804 includes a plurality of tire position buttons 826 from which to select. In one or more embodiments, the app 106 displays a diagram or 3D model of the tire positions to make clear for the user 110 which tire 600 should be scanned to record the data in the correct location. In one or more embodiments, the tire position buttons 826 may be identified by abbreviations (e.g., for the front set of tires, LF for left front and RF for right front and, for the rear set of tires, LFI for left front inner, LFO for left front outer, RFI for right front inner, RFO for right front outer, LRI for left rear inner, LRO for left rear outer, RRI for right rear inner, and RRO for right rear outer). The tire position labels may differ across embodiments based on user or fleet preference. In one or more embodiments, the tire position buttons 826 may include a color-coding system or other visual indicator to denote whether a scan has been recorded for that tire position. For example, a tire position button 826 may be red or have a red border until a scan is recorded during a session, and thereafter, the tire position button 826 may be changed to green or to have a green border. Alternatively or additionally, the tire position button 826 may, for example, include a check mark after completion of a scan and / or an " X" until a scan has been performed.

[0074] After one of the tire selection buttons 826 is selected, the app 106 directs the user to a scanning screen 805 as shown in FIG. 5F. Prior to scanning, the app 106 may prompt the user 110 to ensure that the scanning environment is suitable, with conditions such as adequate lighting (natural or artificial), minimal obstacles or obstructions near the vehicle, safe weather conditions (e.g., no precipitation), and sufficient space to scan all vehicle 125 components without interference.

[0075] The app 106 may provide environmental readiness checks using one or more device sensors, including ambient light sensors, cameras 220, and LiDAR 210. In one or more embodiments, the app 106 notifies the user 110 of insufficient light in the area based on the environment readiness check results or of excessive lens or sensor blur. In one or more1760065464embodiments, the app 106 displays a short video explaining how to prepare the environment for optimal scanning results.

[0076] In one or more embodiments, the scanning screen 805 may include a " Start Scan" button 827. When the user 110 presses the " Start Scan" button 827. the user device 105 begins scanning the tire using the 2D and 3D sensors of the user device 105. After the scan is complete, the user 110 may be directed back to the selection screen 804 as shown in FIG. 5G to select another tire position button 826 until all of the tire position buttons 826 have been selected and scans have been recorded for each tire. Alternatively, in one or more embodiments, the app 106 may auto-assign a tire scanning sequence for efficiency. As can be seen in FIG. 5G, one tire selection button 826 is shown as having a scan recorded for that tire position. As shown in FIG. 5H, all of the tire selection buttons 826 indicate that a scan has been recorded for each of the tire positions. In one or more embodiments, the app 106 may operate in a training mode in which the user 110 also manually enters tread depth measurements. In such embodiments, the system 100 may assign rewards as outlined in the description below to the user 110 for collecting training datasets to better enhance the intelligence of the system 100.

[0077] During the scan, with reference to tire 600 shown in FIG. 8 A, the user 110 positions the device 105 near the tire tread 610 surface, ensuring an appropriate angle and distance as instructed by the app 106, and then the app 106 captures the 3D point cloud data and 2D image data. In certain instances, this data is captured in a very short time frame, e.g., milliseconds, but in other instances, the app may prompt the user 110 to motion the device 105 across the tire tread 610 surface in a back-and-forth or sweeping motion from right side of tire 640 to left side of tire 630 and vice-versa along the tread ribs 612, shoulders 613, and grooves 614 until a significant portion of tire tread 610 is captured. The app 106 will guide the user 110 during the scan and inform the user 110 that the scan is complete when enough data points have been captured. This ensures comprehensive coverage of the tread's 610 surface. The more detailed version of the scan may be employed when scanning other vehicle components, such as a vehicle axle or other large structures.

[0078] In one or more embodiments, the app 106 provides real-time feedback using, e.g., on-screen symbols (e.g., green for correct positioning, red for incorrect alignment), text messages (e.g., " Move device closer" or " Adjust angle for better coverage"), sounds (e.g.,1860065464positive or negative earcons). haptic feedback (e.g., vibrations for incorrect movement), video and animations (e.g., short clips or animations guide the user on how to properly position and move the device for accurate scanning).

[0079] Once properly positioned and moved across the tire 600, the device 105 captures 3D point cloud data 230 and imagery data 240. FIG. 6 depicts the point cloud data 230 and imagery data 240 as captured during a scan. As can be seen in FIG. 6, the imagery data 240 includes a photograph of the tire tread 610, and the point cloud data 230 includes a multitude of points plotted within an x-y-z coordinate system, providing a 3D representation of the tire tread 610.

[0080] After successfully scanning the tires 600 (or other vehicle components) of the tractor 130, the app 106 may direct the user 110 to the vehicle selection screen 802 as shown in FIG. 5B where the second button 823 can be selected to initiate data input for a trailer 140. The app 106 then proceeds to selection screen 803 in which a particular trailer 140 is selected as shown in FIGS. 51 and 5J. The app 106 then directs the user 110 to another component selection screen 804 configured for the trailer 140. In the embodiment depicted in FIG. 5K, the component selection screen 804 includes a plurality of tire position buttons 826 from which to select for the selected trailer. In one or more embodiments, the tire position buttons 826 may be identified by abbreviations (e.g., LFI for left front inner, LFO for left front outer, RFI for right front inner, RFO for right front outer, LRI for left rear inner, LRO for left rear outer. RRI for right rear inner, and RRO for right rear outer).

[0081] After one of the tire selection buttons 826 is selected (or an automatic sequence of tire selections is initiated), the app 106 directs the user to a scanning screen 805 as shown in FIG. 5L. In one or more embodiments, the scanning screen 805 may include a " Start Scan" button 827. When the user 110 presses the " Start Scan" button 827, the user device 105 begins scanning the tire using the 2D and 3D sensors as described above. After the scan is complete, the user 110 may be directed back to the selection screen 804 as shown in FIG. 5M to select another tire position button 826 until all of the tire position buttons 826 have been selected and scans have been recorded for each tire. As can be seen in FIG. 5M, one tire selection button 826 is shown as having a scan recorded for that tire position. As shown in FIG. 5N, all of the tire selection buttons 826 indicate that a scan has been recorded for each of the tire positions.1960065464

[0082] In one or more embodiments, the app 106 ensures all tires 600 of the vehicle 125 (both tractor 130 and trailer 140) are scanned before submitting an inspection report. Once all tires are scanned, the tire inspection is complete. The data is uploaded 270 to the cloud 170 and processed using the data processing pipeline shown in FIGS. 7A and 7B.

[0083] In one or more embodiments, the app 106 analyzes the captured data in real-time to detect, e.g., DOT out-of-service criteria, in particular tire 600 defects, such as tread depth 620 below legal limits, sidewall 650 damage, cuts, exposed cords, uneven wear, or other conditions that could render the vehicle 125 out of service. In one or more embodiments, the app 106 automatically measures the tire tread depths 620 for each tire 600 and detects w ear patterns based on TMC's Radial Tire Conditions Analysis Guide, prepared by American Trucking Association, which is incorporated herein in its entirety by reference thereto. If an out-of-service criteria is detected, the app 106 alerts the user 110 immediately with visual, audio, and text notifications, describing the issue and its implications. Such alerts may include video guidance in which a video tutorial is displayed, explaining how to fix or address the detected DOT issue. Additionally, in one or more embodiments, alerts are sent to the fleet personnel 150 on the backend, notifying relevant personnel 150 of the issue for prompt resolution.

[0084] If the scan is a general vehicle inspection scan, the process proceeds in substantially the same way. In particular, the user 110 selects one or more vehicle 125 components for scanning. Examples of vehicle components that may be scanned using the app 106 include, but are not limited to, rims, axles, brakes, bumpers, fenders, lights, doors, engine, suspension, steering components, chassis, floor, roof, pedals, seats and seatbelts, airbags, windows, wires and connections, batteries, computers, charging systems and ports, safety systems, electronic control devices, and dash, including displays and dash lights. For vehicles 125 with multiple segments (e.g., tractor-trailers), components can be selected for each segment individually. The app 106 displays a diagram or list of selected components, allowing the user 110 to verify and adjust the selection.

[0085] Similar to the discussion above, for each vehicle 125 component displayed, the user 110 selects a component to scan and presses the " Start Scan" button to start the component inspection scan. The user 110 positions the device 105 near the selected component to initiate the scan. The scanning process involves capturing 3D point cloud data2060065464230 and imagery data 240 of the component surface and analyzing the data for DOT Out-of-Service Criteria as outlined in the North American Standard Out-of-Service Criteria, prepared by the Commercial Vehicle Safety Administration (CVSA), which is incorporated herein in its entirety by reference thereto. Such conditions include brake wear exceeding thresholds, damaged axles, defective lights, loose or damaged trailer doors, and other defects causing out-of-service violations.

[0086] The CVSA writes a new North American Standard Out-of-Service Criteria handbook each year. Certified commercial vehicle safety enforcement personnel rely on the North American Standard Out-of-Service Criteria to assess whether a commercial motor vehicle 125 or its driver 110 poses an immediate hazard and should be taken out of service. For brake systems, the out-of-service criteria include defective brakes, including missing components, audible air leaks, brakes out of adjustment, etc. For steering mechanisms, the out-of-service criteria include excessive steering wheel play or other defects affecting vehicle control. For tires and wheels, the out-of-service criteria include tires with insufficient tread depth, significant damage, or improper inflation as well as damage or missing wheel components. For lighting devices, the out-of-service criteria include inoperative required lights, such as headlamps, tail lamps, stop lamps, or turn signals. For suspension systems, the out-of-service criteria include cracked or broken leaf springs, missing suspension components, or other defects compromising vehicle stability, and for coupling devices, the out-of-service criteria include defective or improperly secured fifth wheels, pintle hooks, or other coupling devices.

[0087] If an issue is detected, the app 106 alerts the user 110 with a detailed description and recommended corrective actions. The app may provide video guidance in the form of short tutorial clips to demonstrate how to fix or address the specific issue detected. Additionally, alerts may be sent to the fleet personnel 150 on the backend, notifying relevant personnel 150 of the issue. The app also guides the user 110 using the same feedback mechanisms as in tire 600 scanning (e.g., symbols, text, sounds, haptics, and videos). Additional details, such as component type, condition, or specific measurements, can also be manually input to further characterize the issue or any action taken to remedy it.

[0088] The app 106 verifies that all selected components for the vehicle 125 are scanned before submitting the inspection report. Once all components are scanned, the vehicle 1252160065464component inspection is complete. The data is uploaded to the cloud 170 and processed using the data processing pipeline shown in FIGS. 7A and 7B.

[0089] In one or more embodiments, the user 110 uses an eDVIR application that is integrated with the system 100 to perform pre-trip and post-trip inspections. In such embodiments, the eDVIR application uses the system 100 for vehicle 125 component video inspection, 3D scanning, data storage, data analysis, and optional cloud 170 data upload functionality. The app 106 may be launched or invoked by the eDVIR application (e.g., via an SDK integration) to perform the scanning and analysis functionalities described herein, and the resultant datasets may be transferred back to the eDVIR application after the vehicle data is scanned and processed by the app 106. In one or more embodiments, the user 110 can alternatively use the app 106 to directly complete an eDVIR report using API integrations with the eDVIR provider. In such embodiments, the app 106 uses Al to automatically generate the eDVIR report based on the scanned vehicle data.

[0090] Alternatively or additionally, the app 106 may be used by the eDVIR inspection app to scan tires and any other relevant vehicle components implemented through a third-party integration, and the data may be uploaded to the cloud backend 170 where the data is processed and stored. In one or more such embodiments, the SDK for the app 106 supports processing either on-device or via the cloud backend 170. If an on-device processing case, the data can be provided back to the calling eDVIR app directly through the integration code and / or via API 300, such as the eDVIR API 300F. In a cloud-processing case, the eDVIR application and / or an eDVIR provider backend receives the processed outputs and / or inspection datasets from the backend system 170 of the system 100 via an API integration 300, such as the eDVIR API 300F.

[0091] The user 110 can complete full eDVIR pre-trip and post-trip inspections using the app 106. During the inspection, the user 110 can scan all components of the vehicle 125, such as those provided in the exemplary list above. After scanning each item, the app 106 will automatically start creating the eDVIR report in the background. Once all items are completed, the final eDVIR report is generated and provided by the app 106 to the eDVIR provider via an application integration and / or an API integration. Notes regarding the tire 600 condition, including tread depth 620, wear patterns, and maintenance suggestions about the tires 600 as described above, may be included in the generated eDVIR report to provide highly2260065464accurate and detailed information for fleet maintenance and safety teams, and for compliance and inspection purposes by DOT, FMCSA, and law enforcement.

[0092] In one or more embodiments, data from each scan is processed locally on the device 105 and / or uploaded to the cloud 170 for advanced analysis. In one or more embodiments, the data processing pipeline shown in FIGS. 7 A and 7B processes the data. Algorithms analyze the data, including point cloud processing, image processing, Al models, and DOT compliance analysis. Point cloud processing may involve filtering, segmentation, clustering, surface modeling, and feature extraction. Image Processing may involve enhancing captured images for better quality, detecting features or defects, and overlaying imagery data on 3D point clouds. Al Models utilized may include convolutional neural networks, GPT-based models, and anomaly detection algorithms to derive insights. The DOT compliance analysis involves automatically comparing scanned data against DOT out-of-service criteria to flag defects.

[0093] Having described the app interface for conducting the scans, FIGS. 4 A and 4B depict process flow diagrams for data collection and analysis of the scanned component 205 of the vehicle 125. In one or more embodiments, the 3D data is generated by a 3D sensor 210, and the 2D data is generated by the 2D sensor 220. However, in one or more other embodiments, the 3D data can be generated instead from imagery data 240 of the 2D sensor 220. In such embodiments, a 3D reconstruction 250 can be made from the imagery data 240 to produce point cloud data 230.

[0094] The point cloud data 230 and imagery data 240 are stored on device storage 260. In one or more embodiments, the user data 200, point cloud data 230, and imagery data 240 are uploaded to the backend system 170 via Wi-Fi or cellular internet connection. In the backend system 170, the data 200, 230, 240 is stored in cloud storage 171.

[0095] As mentioned, in one or more embodiments, the device 105 captures imagery data 240 during the vehicle 125 inspection process. In one or more such embodiments, the imagery data 240 is captured locally on the device storage 260 and may then be transmitted to the cloud 170 for data retention, proof of inspection, machine learning, and legal purposes. The imagery data 240, in particular any video file produced, may also be associated with database records on the device 105 including timestamp, ID of the user 110, ID of the vehicle 125, user input data 200, and location coordinates so that the data can be searched quickly and indexed 2360065464for quick integration and lookup from the app 106 and web platform 161. The database records on the device 105, including user input data 200, are stored and uploaded alongside the imagery data 240 and point cloud data 230 from the device 105 to cloud storage 171.

[0096] In one or more embodiments, the device 105 processes the scanned data 230, 240 locally using integrated Al, point cloud processing algorithms, image processing algorithms, and other computational techniques tailored for user devices 105, e.g., for mobile device 120 or XR device 180. This enables offline functionality and real-time feedback.

[0097] FIGS. 7A and 7B depict an embodiment of the data processing pipeline used to derive valuable insights from the vehicle scan data. In FIG. 7A, the data pipeline starts with raw point cloud data 230 and raw RGB and RGB-D imagery data 240 as inputs. The RGB-D data contains depth data that can be used to reconstruct a 3D model of the components. The branches for processing point cloud data 230 and imagery data 240 are split out initially and described in detail below.

[0098] Image processing may take place according to a 3D Point Cloud 230 Processing Branch. The 3D point cloud data 230 progresses through its own branch of the data pipeline where it is filtered for noise 500 and checked for objects via 3D object detection and classification 505. Based on the results of 505, a determination 510 is made whether valid objects were detected in the scan data. Objects are considered valid if they match a list of supported vehicle components as listed above. An example of invalid scan data is when a user 110 scans a tree using the app 106, and the system 100 detects the tree and tells the user 110 that the scan is invalid in the app 106 and does not progress further on processing the invalid scan data. In one or more embodiments, invalid scan data is deleted by the app 106 automatically. For all valid objects detected in step 510, these objects are further processed and analyzed in the data processing pipeline.

[0099] Image processing also takes place according to an RGB and RGB-D Imagery 240 Processing Branch. The raw RGB and RGB-D imagery data 240 flows through its own branch of the data processing pipeline. It undergoes image noise filtering 520, object detection and classification 525, and valid scan object determination 530. The objects are considered valid if they match a list of supported vehicle components as included in Maintenance Scan Items and DOT Out-of-Service Criteria sections. For all valid objects detected in step 530, these objects are further processed and analyzed in the data processing pipeline.2460065464

[0100] As shown in FIG. 7A, the valid 3D point cloud objects 510 and RGB imagery objects 530 are fused together using a data fusion process 535. The data fusion process 535 entails spatially registering the datasets using object position, features such as landmarks, and camera frame reference information. Once the datasets are fused, the combined dataset comprising 3D point cloud 230 and imagery 240 data can be used for detailed and accurate data analysis.

[0101] In step 515 of FIG. 7A, for each valid object with both 3D point cloud 230 and imagery’ 240 data, the state A is executed. In FIG. 7B, the state A is followed by the following data operations, which are performed using the fused 3D point cloud 230 and imagery 240 datasets as needed:

[0102] As discussed above, embodiments of the presently disclosed method can be utilized even if the object is a vehicle component that is not a tire 536. In such methods, the disclosed method and system begin with feature detection and extraction 540. This step is where useful features of the object are detected for downstream processing and analysis. These include features that stand out on the object such as defects, bends, scratches, cracks, or discoloration. Thereafter, the defects are detected and classified in step 545. In this step 545, defects are detected based on the features detected and extracted earlier in the previous step 540. Defects are classified based on known defect features, images, models, and descriptions, including defects presented in, but not limited to, ATA’s TMC Radial Tire and Disc Wheel Digest, ATA’s TMC Radial Tire and Disc Wheel Service Manual, and ATA's TMC Recommended Practices Manual.

[0103] In one or more embodiments, if the object is atire 537, then the method and system begin with tread pattern detection 550. In this step 550, a tire tread pattern and one or more tread wear patterns are detected. The tire tread pattern may be detected using a pre-existing tire tread template dataset. The system and method may detect and / or classify tire tread wear patterns based on one or more ATA TMC reference materials, including, for example, the TMC Radial Tire Conditions Analysis Guide, the ATA TMC Radial Tire and Disc Wheel Digest, the ATA TMC Radial Tire and Disc Wheel Service Manual, and the ATA TMC Recommended Practices Manual, each of which is incorporated herein in their entireties by reference thereto. In one or more embodiments, user-trained Al models trained using the industry-standard reference materials described above may be used to detect tread wear2560065464patterns. In other embodiments, user-trained Al models may be used to detect the wear patterns.

[0104] After tread pattern detection 550. the tread depth is measured in step 555. In this step, the tire tread depth 620 is measured as shown in FIG. 8A. As can be seen there, measurements are computed by taking one or more points on the top 615 of a given tread block 611 or tread shoulder 613 and one or more points at the bottom 616 of the same block 611 or shoulder 613, and then the distance 620 between the top 615 and bottom 616 points is computed. The tread depth 620 is equivalent to the depth of the tread grooves 614 from the top 615 to the bottom 616 of the tread 610 at one or more measurement locations.

[0105] The system 100 detects points at the top 615 and bottom 616 of the tread in the scan datasets 230, 240 (e.g., as shown in FIG. 6) using models and algorithms listed in the Data Processing Models and Algorithms section below. The system 100 then computes the tread depth 620 using those detected points as described. A visualization of this is provided in FIG. 8A, where a rear-facing view of a tire tread 610 is shown with the left shoulder 613 on the left side of the tire 630 and the right shoulder 613 on the right side of the tire 640. The point locations 660, including top 615 and bottom 616 points, may be averaged, clustered, or handled separately from each other in various combinations or groupings in order to derive the depth measurement 620, and the depth measurement 620 may be one or more values including an average, statistic, or aggregation, such as min or max, of the multiple distance measurements for the various combinations of top 615 and bottom 616 points.

[0106] Referring to FIG. 8B, the sample locations 660 where tread depth measurements 620 are taken on a tractor trailer tire 600 by the system 100 can be seen. The distance between points is measured in the 3D x-y-z coordinate system and converted to millimeters using the formulas necessary based on parameters used to calibrate the device sensors as well as any device 105 or sensor-specific parameters. Multiple depth measurements 620, as described above, are taken across the tire 600 surface in order to ensure accuracy in the overall tire tread measurements 620 provided to the downstream processes and users.

[0107] Conventionally, these measurements 620 are taken manually by drivers 110 and maintenance technicians 110 using a tire gauge in a limited number of spots on a given tire 600. The measurements are written on paper and manually entered into a computer system at a later date. These manual actions limit data analysis and sharing capabilities and are prone 2660065464to error. Further, the limited manual measurements 620 per tire 600 give the maintenance technicians 110 and fleet website users 150 an imperfect view on the health of the tires 600.

[0108] The embodied system 100 provides an automated and accurate solution for measuring the tire tread depth 620 using multiple data points 660 from across the scanned surface of the tire 600. It provides the app users 110 and fleet maintenance teams 150 with a more holistic view of the tire 600 health and wear rates of their vehicles 125. It also provides a more accurate representation of DOT out-of-service criteria based on the tire tread depth 620.

[0109] Referring again to FIG. 8A, the tread ribs 612 are columns composed of blocks 611 on the tire tread 610. The tire sidewall 650 is also shown. In one or more embodiments, tread pattern analysis 560 involves analyzing the tire tread 610 pattern based on, but not limited to, information in the TMC's Radial Tire Conditions Analysis Guide.

[0110] FIG. 9 shows a more in-depth view of the tire wear analysis pipeline flow. First, the system performs a search for the given tire 600 by tire ID or tire position 700. If the tire does not exist in the system 705, then there is not enough data to perform wear rate computation 710 or detect wear patterns developing over time. If it is true that there is not enough historical tire data to compute a wear rate 710, then the tire 600 scan data is retained in cloud storage 171 and kept for future comparison and analysis. In the future, if the same tire 600 is scanned again, then tire 600 wear can be computed based on historical data comparison for that tire 600. If the tire does exist in the system 715, then the system 100 gets the previous tire scan dataset including point clouds and imagery 720 as well as the current tire scan dataset 730.

[0111] In step 740 of FIG. 9, the current and previous tire 600 scan datasets are aligned and compared with each other. The system 100 gathers data for the factors influencing tire-related calculations 750. The results of the tire data comparison 740 are joined with the influential factors influencing tire-related calculations 760. Based on the comparisons, the system 100 calculates the wear rate 770 and performs wear pattern recognition 780.

[0112] In FIG. 9, the tire wear rate and wear patterns are analyzed 790 to derive important information about the tire 600 and vehicle 125. The wear pattern recognition and analysis are based on historical data comparison as well as information such as patterns, features, images,2760065464models, descriptors, and factors presented in the TMC Radial Tire Conditions Analysis Guide and other TMC guides and manuals mentioned herein. The output of the analysis includes wear analysis results, reporting, and maintenance suggestions 795 that are provided to downstream processes, APIs 300, and users (110, 150) of the system 100.

[0113] In FIG. 7B, the datasets and results from 545 and 555 are provided to the DOT Out-of-Service Detection and Classification module 575. Based on the results from 575 and using the criteria outlined in the DOT Out-of-Service Criteria section, the system 100 determines if DOT Out-of-Service notifications 585 are generated for the fleet users (110, 150). Maintenance suggestions 580 may also be made based on the vehicle 125 component defects, tire 600 measurements, or issues that were found.

[0114] The datasets and results from 545, 555, and 560 in FIG. 7B are also provided to compute the wear rate, wear patterns, wear prediction, and correlation of wear between vehicle components including tires 565. The resultant datasets are further analyzed 570, and insights, scan results, and reports 590 are generated for users (110, 150) and APIs 300 based on the results. The app users 110 receive the information via the device 105 and app 106. The fleet users 150. including management, safety, and maintenance, receive the information via the web platform 161. The APIs 300 receive the data from the device 105 or from the cloud backend 170, depending on where the data processing takes place.

[0115] Maintenance suggestions 580 are derived from analysis 570 results and DOT Out-of-Service detection and classification results 575. The maintenance suggestions 580 and DOT Out-of-Service detection and classification results 575 are provided with the insights, scan results, and reports 590 to the downstream users (110, 150) and APIs 300.

[0116] Continuing with FIG. 7B, the maintenance suggestions 580 and DOT Out-of-Service notifications 585 are provided to the users (110, 150) via the notification service 430. The notification service 430 offers push notifications, emails, chat messages, text messages, etc. to deliver notification messages to users (110, 150). Notifications may vary in priority. The priority scale in one or more embodiments includes multiple values from low to critical, such as low, medium, high, and critical. Users (110, 150) may opt in or out of receiving messages of varying priorities based on preference. The messages may also have other groupings such as vehicle component type or notification topic category, such as safety, maintenance, driver manager, management, etc. Users (110, 150) may opt in or out of 2860065464receiving messages from the various topic groups and categories based on their preferences and roles in the company.

[0117] WEB PLATFORM

[0118] Hosted in the cloud 170, the web platform 161 provides tools for fleet maintenance teams (110, 150) to manage and analyze vehicle 125 data, offering tire and component health dashboards, historical data and reports, data correction and validation, and real-time alerts, amongst other possibilities.

[0119] With respect to the tire and component health dashboards, the dashboards summarize health metrics for all tires 600 and vehicle 125 components. The dashboards highlight critical issues requiring immediate attention, such as DOT out-of-service violations or low tire tread. FIG. 10 showcases a fleet vehicle 125 maintenance dashboard focused on tire and vehicle 125 component health. In one or more of the dashboards 190, a tire 600 health overview 191 is given with important alerts at the top, such as brake alerts 192. Key maintenance items are highlighted and displayed for the fleet users 150 in the dashboard 190 based on their preferences, topic subscriptions, and priorities, which are configurable in the system 100. The users 150 can also create and configure their own custom dashboard pages using widgets in the dashboard creator component of the web platform 161. The users 150 can also create and configure their own custom widgets using available datasets in the widget creator component of the web platform 161.

[0120] In one or more dashboards, such as shown in FIG. 10, fleet users 150 are presented with critical video inspection results and 3D scan inspection results of critical or highly important maintenance, safety, and DOT out-of-service items that they have to act on in a timely fashion in order to avoid drivers 110 or vehicles 125 going out of service for any period of time. The web users 150 are alerted when these occur via the notification service 430 and the items are displayed on the web platform 161 dashboards for them to review.

[0121] With respect to the historical data and reports, users 150 can view past scans, wear patterns, and maintenance histories for the entire fleet on the web platform 161. Additionally, users 150 can generate advanced reports tailored to fleet maintenance and safety needs. Several advanced reports are detailed below.2960065464

[0122] With respect to data correction and validation, the web platform 161 provides tools for refining and validating scanned data. Real-time alerts can be provided through the web platform 161 for, e.g., DOT compliance violations and critical maintenance needs.

[0123] In one or more embodiments, web reports can be generated for fleet maintenance and safety. One example of such a report is a tire and vehicle component health overview, which summarizes the health status of all vehicle 125 components in the fleet, including tires 600, brakes, and lights. The report highlights items with critical wear or DOT out-of-service criteria and projects component life expectancy based on current wear rates. Additionally, in one or more embodiments, the web report may provide wear pattern analysis based on visualization of tread 610 and component wear patterns across the fleet. Still further, in one or more embodiments, the web report can identify alignment issues, overloading, or maintenance lapses causing uneven wear.

[0124] The w eb report may further include a DOT compliance report as part of the report or as a separate report, which lists detected DOT out-of-service issues, grouped by severity and provides maintenance recommendations to bring components into compliance.

[0125] Using these reports, a user 110 or fleet management personnel 150 can gain insight into fleet safety. Vehicle 125 health data can be correlated with safety metrics, such as braking efficiency and accident risk, to offer actionable recommendations to enhance fleet safety and compliance. Such insights can be used to generate a cost optimization report to quantify potential cost savings from proactive maintenance and calculate financial impacts of delayed repairs or replacements. Additionally, the insights can be used to prepare a maintenance scheduling report in which optimized maintenance schedules are recommended based on fleet-wide health data. The maintenance schedules can be synchronized with telematics and maintenance systems for seamless task management.

[0126] The user 110 or fleet personnel 150 can create customizable fleet reports. Such reports allow fleet managers 150 to create tailored reports focusing on specific metrics or compliance needs.

[0127] As mentioned previously, the web reports may be generated using information obtained through integration with one or more third-party APIs, enabling the collection and exchange of data with existing third-party systems. FIG. 11 depicts APIs 300 integrated with 3060065464the web platform 161 via the backend system 170. To enhance data accuracy, incoming data may be retrieved from relevant information provided by the third-party APIs 300, and the web platform 161 may remain synchronized with current data feeds to ensure data consistency.

[0128] In addition to receiving data from third-party systems, the web platform 161 may also share data including scan results, inspection results, maintenance recommendations, and compliance insights 590 with one or more third-party APIs and systems 300, including the systems 300A-300J shown in FIG. 11, as well as any other compatible third-party databases, systems, and APIs depending on fleet use case.

[0129] This system 100 integrates cutting-edge scanning, data processing, DOT compliance checks, and fleet management tools to optimize vehicle 125 maintenance, routing, and safety. Advanced features like video tutorials, real-time alerts, and detailed web reports ensure ease of use, actionable insights, and proactive fleet management. By addressing both vehicle 125 and tire-specific 600 maintenance, compliance, and safety needs, it provides a comprehensive, scalable solution for modern fleet operations.

[0130] DATA PROCESSING MODELS AND ALGORITHMS

[0131] One or more of the following AI models or algorithms may be used in one or more of the data processing pipelines used to interpret and analyze the 3D vehicle 125 point cloud data 230 and imagery data 240. Third party data collected from APIs, such as telematics and maintenance APIs, may also be used in combination with the point cloud 230 and imagery 240 data to derive more value for the fleet users 150. The models may be trained and run in inference mode on CPUs, GPUs, or any other processing units capable of machine learning. The AI models or algorithms include at least one of transformer models; convolutional neural networks, in particular for feature extraction and depth analysis; multilayer neural networks; large language models (LLMs) for constructing knowledge bases, querying datasets, and generating vehicle maintenance recommendations; generative Al models (e.g., generative transformer models or generative adversarial networks (GAN)); recurrent networks; reinforcement learning; supervised learning; unsupervised learning; clustering; classification models; regression models for, e.g., wear prediction and maintenance scheduling; 3D surface reconstruction models including; time-series models; models trained via transfer learning; or3160065464multi-modal models. DOT compliance analysis can be conducted using algorithms to identify issues causing out-of-service violations and recommend corrective actions.

[0132] Point cloud processing algorithms may be used in one or more of the data processing pipelines used to interpret the 3D point cloud data 230 and imagery data 240, including, for example, filtering, segmentation, clustering, classification, registration, reconstruction, surface modeling, feature extraction, normal estimation, contour detection, object detection, object tracking, scene understanding, anomaly detection, change detection, compression, visualization, calibration, or a combination of one or more of the foregoing.

[0133] Image processing algorithms may be used in one or more of the data processing pipelines used to interpret the 3D point cloud data 230 and imagery data 240, including, for example, edge detection, segmentation, image transformation, morphological operations, feature detection and extraction, image filtering, image registration, optical flow estimation, color space transformation, convolutional neural networks, or a combination of one or more of the foregoing.

[0134] FIG. 12 provides a summary of the end-to-end process of the data preparation and model training pipeline 1000. Broadly, the end-to-end process involves the steps of collecting labeled data from a training version of the tire or vehicle component scanning app, creation of standardized datasets, training and validation splitting, feeding the datasets into the multimodal model, performing training and validation loops, selection of the appropriate model, and deployment of the trained model to the API interface.

[0135] Referring to the first block 1001, the application acquires the captured data as well as the manual input from the technician. In particular, the technician enters manual tread depth, which provides the ground truth label. Additionally, the technician may enter tire metadata, including visible tire wear patterns and conditions, and vehicle metadata. As discussed above, the app 106 captures depth frames and RGB images, reconstructed RGB point clouds, timestamp, and identifiers. The data acquisition by the app 106 is described additionally below in relation to FIG. 13.

[0136] In the second block 1002, the datasets are uploaded and stored in the backend system 170. In one or more embodiments, the app 106 creates point clouds as binary files (.bin) and together with the RGB images (.heic / .png) and session identifiers compresses the 3260065464scan data into a ZIP archive. The ZIP archive is then uploaded to cloud storage (e.g., Google Cloud Storage or an equivalent). The app 106 receives an object URI or other storage path identifier. Additionally, metadata regarding the vehicle and / or tire is uploaded to the backend database (e.g., Firestore or an equivalent). As mentioned above, the metadata may include a vehicle identifier, a tire position, a tire make / model / size, manual tread depth label (in the training mode or version of the app 106), wear pattern, tire condition, mileage, technician ID, timestamp, and storage path (URI) of the ZIP file. The ZIP file containing the raw sensor data and the database document containing the structured metadata and label provide a fully supervised training example. Portions of the actions performed in the second block 1002 are also shown and described in relation to FIG. 13.

[0137] In a third block 1003, the backend data preparation pipeline is summarized. A dataset is extracted by unzipping the dataset, parsing point clouds, decoding images, loading labels, and loading metadata. Thereafter, the point clouds are filtered and isolated through color masking, outlier removal, clustering (e.g., DBSCAN), downsampling (e.g., 4096 points), and normalizing coordinates. This process is described more fully below in relation to FIG. 14. Further, the RGB images are processed by reading the image file, resizing and normalizing the image, and optionally performing multi-frame averaging. Next, the dataset is assembled. During dataset preparation, the system extracts the technician-entered tread-depth label and pairs it with the filtered and downsampled point-cloud and image samples generated from each uploaded scan. In one or more embodiments, an NPZ dataset is created, which contains X_points of the normalized point clouds, X_images of the processed images, y labels of the technician-entered measurements, and metadata, such as tire position, brand, etc. The NPZ format is a binary file format used to efficiently store multiple NumPy arrays. The output of this third block 1003 of the process 1000 is a cleaned supervised machine learning training dataset. The cleaned dataset is represented in FIG. 18, which is described more fully below.

[0138] With reference to the fourth block 1004, the datasets are split for training and validation. A variety of split ratios may be employed, such as using 80 percent of the datasets for model training and the remaining 20 percent for model validation. Advantageously, this process ensures that there is no data leakage between the splits. The splits may be made using random shuffling or stratified sampling, for example. In one or more embodiments, the validation set is reserved for monitoring model performance and generalization during 3360065464training. The outputs of the fourth block 1004 of the process 1000 are training datasets and validation datasets.

[0139] A fifth block 1005 of the process 1000 is model feeding and training loop. In this block, a multi-modal model is initialized by loading the point-cloud encoder, loading the pretrained image encoder, initializing fusion layers, and initializing regression head. A training loop is then performed, and each epoch involves a forward pass over the image processing branch and point-cloud branch as described below in relation to FIG. 14. A prediction is computed and compared against technician-entered labels. The loss is computed, e.g., using mean-square error (MSE) or similar regression objectives, batch training with stochastic gradient descent (SGD) or Adam optimizers, or early stopping or checkpointing. Gradients are backpropagated, and parameters are updated (e.g., using an Adam optimizer). After each epoch, a validation loop is performed in which the model is evaluated on a validation set and a validation loss is computed. This enables the tracking of a best-performing model.

[0140] In a sixth block 1006, a model is selected and deployed for use by the app 106. The best model as evaluated in the previous block is saved and trained model weights are exported. The model is uploaded to a registry or cloud storage, and a version identifier is assigned for tracking and rollback, if needed. The model is then deployed for the API interface. In particular, the selected model is deployed to an inference environment, such as Cloud Run, a GPU / CPU compute instance, or an edge device. A secure API point is exposed for acceptance of scan data and to return automated tire-analysis outputs in real time or near-real time to user devices 105. For example, the inference API may provide tread-depth measurements, wear-pattern classification, detection of irregular or uneven wear, sidewall or tread defect indicators, indicators of potential under-inflation based on sidewall deformation, and / or additional analytical and predictive outputs generated by the trained model. Production applications and external consumer systems retrieve these results through the inference API for integration into downstream workflows.

[0141] FIG. 13 depicts a flow diagram 1100 outlining multi-frame data capture and pointcloud reconstruction. In one or more embodiments, with reference to FIG. 13, the app 106 acquires a depth map 1101 by utilizing a sequence of frames in which each frame may include a depth map 1101 obtained from the device’ s depth-sensing subsystem, a corresponding RGB3460065464image 1103, pose information representing the device’s position and orientation 1105 in world coordinates 1106, and optionally simultaneous localization and mapping (SLAM) feature points provided by the tracking subsystem.

[0142] In one or more such embodiments, the app 106 reconstructs a point cloud 1102 during or immediately after each frame capture using the depth map 1101, camera intrinsics, and RGB image 1103. In one or more embodiments, this reconstruction is performed entirely in software and yields an RGB-colored point cloud 1104 representing a partial three-dimensional sampling of the tire tread surface.

[0143] In other embodiments, a point cloud may be provided directly by the device’s operating system, depth-sensing hardware, or a system-level API, and the application may utilize such system-generated point clouds either in place of or in combination with software-reconstructed point clouds. Both approaches are compatible with the downstream processing pipeline and may be selected based on device capabilities, performance considerations, or implementation preferences.

[0144] The application may capture a configurable number of frames (for example, 1, 3, 5, 10, or more), accumulating a number of points across the frames 1107. In one or more specific embodiments, five frames are acquired sequentially, although this value may be increased or decreased without limiting the functionality of the system. Each captured frame produces a corresponding partial three-dimensional representation of the tire tread, and the system may fuse multiple frame-level point clouds into a consolidated multi-frame reconstruction.

[0145] Quality-control mechanisms may be employed to discard frames or point-cloud segments exhibiting insufficient depth confidence, excessive motion blur, or other indicators of reduced fidelity.

[0146] In one or more embodiments, the app 106 may also prompt the user 110 to perform a sidewall scan. The sidewall scan may capture information such as tire brand, model, size, or DOT serial number, and may support identification of defects including bulges, cracks, cuts, abrasion patterns, or deformation suggestive of under-inflation or structural fatigue.3560065464

[0147] In one or more embodiments, a three-dimensional point cloud is reconstructed in reconstruction step 1102 from depth-map 1101 data provided by a mobile device's LiDAR or depth-sensing subsystem, for example via ARKit's sceneDepth interface, and camera intrinsics parameters obtained from the mobile device's imaging subsystem, for example via ARFrame camera intrinsics. For each video frame, the system obtains a depth map D and an associated confidence map C, each having dimensions Wd x Hd. Only depth pixels whose confidence value meets or exceeds a minimum threshold (e.g., C(x,y) ≥ cmin) are used in the reconstruction process.

[0148] Let a depth pixel be located at coordinates (xd, yd), where:xd∈ [0, Wd- 1], yd∈ [0, Hd- 1].

[0149] The corresponding depth value is:z = D(xd, yd),

[0150] and pixels with z = 0 or insufficient confidence are discarded.

[0151] In order to normalize and map into the camera image space, the depth-map coordinates are first normalized into the interval [0,1]:

[0152] These normalized coordinates are then scaled into the full resolution camera image space (Wi x Hi), yielding:u = un■ Wi, v = vn- Hi.

[0153] The resulting pixel location (u, v) is used for geometric back projection as well as RGB sampling from the captured camera image.

[0154] Intrinsic calibration can be used to create a back-projection into camera space. ARKit provides the calibrated intrinsic matrix? of the device's camera:0K = 0 fyL0 03660065464

[0155] A homogeneous screen-space sample is formed:Pscreen=LU

[0156] Geometric back-projection is performed by multiplying with the inverse intrinsics and scaling by the depth value:pr>cam ^screen

[0157] Expanding this yields the camera space coordinates:Ycam= z(v - cy) / fy7L^cam-11

[0158] This yields a 3D point located in the coordinate frame of the device camera for the given AR frame.

[0159] Because ARKit internally uses a right-handed coordinate system aligned to the device orientation, additional transformations are applied to ensure correct world-space alignment in portrait mode. These transformations include:

[0160] (a) YZ axis Flip1 0 0 O’0 -1 0 00 0 -1 0.0 0 0 1.

[0161] (b) Rotation of π / 2 Radian About the Z-Axis’0 -1 0 O’1 0 0 00 0 1 0.0 0 0 1.

[0162] The combined portrait orientation correction is:^portrait Rn / 2‘3760065464

[0163] ARKit provides the camera view matrix V, and the world transform 1106 of the camera is computed as:Tcam→world= V-1· TportraitTcam→world= V-1· Tportrait

[0164] Each camera-space point is expressed in homogeneous form:■v^camph K:am■cam 7^cam. 1.

[0165] The transformation into world coordinates 1106 is:ph _ 'T ph"world1cam→world cam >

[0166] and normalized as:J r ■'Y‘world^world=^world ■W ryworld-

[0167] This produces a fully world-aligned point cloud representation suitable for multiframe aggregation.

[0168] The system samples per-point RGB values from the camera image buffer:

[0169] where I denotes the captured image frame. Each 3D point is therefore associated with the corresponding color observed at the same pixel location, yielding a colorized point cloud:(Xworld, Yworld, Zworld, R, G, B).

[0170] This RGB point cloud may be stored in memory, fused with additional frames, or written directly to persistent storage as part of a multi-frame burst.

[0171] In embodiments using multiple depth frames, each reconstructed point cloud is aligned into a common coordinate frame using the world transform described above. Because3860065464ARKit provides temporally stable camera poses, consecutive world-space point clouds may be aggregated without requiring iterative alignment, thereby forming a denser and more complete representation of the tire tread surface. Multi-frame accumulation enhances spatial resolution and robustness, particularly in situations where the depth map produces sparse or noisy observations on certain surfaces.

[0172] Optionally, in one or more embodiments, sparse feature points provided by ARKit's SLAM tracking subsystem may be combined with the depth-derived point cloud. These feature points may help stabilize reconstruction, may supplement areas with limited depth map reliability, and may serve as geometric anchors for multi-frame consistency.

[0173] A feature point with 3D coordinates fi= (Xi, Yi, Zi) may be colorized using projection into the image plane:(ui, vi) = Π(Ki),

[0174] where n( ) denotes perspective projection followed by division by homogeneous coordinates.

[0175] Colorized feature points may be appended to, or blended with, the primary point cloud, yielding a fused representation that incorporates both dense LiDAR geometry and sparse but high-confidence visual features.

[0176] The final output 1108 of this reconstruction process is a world-aligned, colorized point cloud for each frame or for the aggregated sequence of frames. Each point contains both geometric and photometric information, enabling downstream operations such as clustering, tire surface extraction, orientation normalization, wear pattern analysis, and multi-modal machine-learning inference.

[0177] After the desired number of frames have been captured, the app 106 processes each point cloud to remove outliers and prepares the point cloud for backend processing. The system may optionally apply threshold depth confidence values, remove distant or irrelevant points, normalize coordinates, and compress associated imagery data.

[0178] After assembling the point clouds and RGB images, the system compresses 1109 these files into a ZIP archive and uploads 1110 the ZIP to a cloud-based object storage service.3960065464An example of a commercially available cloud-based object storage service suitable for use in embodiments of the present disclosure is Google Cloud Storage. In parallel, the application transmits the associated tire and vehicle metadata — such as vehicle identifier, tire position, tire make and model, tire size, manual tread measurement (in the training version), mileage, technician identifier, timestamp, and the storage path of the uploaded ZIP file — to a cloud datastore. An example of a commercially available cloud datastore suitable for use in embodiments of the present disclosure is Firestore, available from Google. This separation enables the ZIP archive to focus solely on raw sensor data while the metadata is efficiently stored, indexed, and retrieved through a structured database. Together, the metadata entry and the ZIP archive constitute a complete labeled training sample.

[0179] Vehicle- and tire-related metadata may originate from manual user entry, telematics integrations via API. or computer-vision extraction from sidewall images, amongst other possibilities.

[0180] Upon receiving a dataset from the app 106, the backend pipeline extracts and interprets the files as shown in FIG. 14. Using customized dataset parsing procedures, the system may parse point clouds from binary files, decode HEIC or PNG images into RGB arrays, normalize color values, align and normalize coordinate systems, and / or extract labels such as ground-truth tread depth (training version only). In one or more embodiments, the system supports multiple binary formats for backward compatibility with earlier scanning app versions.

[0181] As shown in FIG. 14, the data processing pipeline 1200 describes point-cloud filtering, tire surface isolation, and canonical preparation. A first step 1201 in the pipeline 1200 is to load the reconstructed point cloud including the XYZ and RGB values. In one or more embodiments, a second step 1202 of applying a filter to the values is performed to remove invalid or non-finite values. Additionally, a third step 1203 of color-based tire rubber masking may optionally be performed on regions of low brightness or low saturation to isolate dark rubber. This may be done to remove background points corresponding to, e.g., shop floors, vehicle bodies, or environmental clutter. In a fourth step 1204, statistical spatial outlier values are removed using nearest-neighbor Z-score trimming or median absolute deviation (MAD) to trim distant or spurious points.4060065464

[0182] In a fifth step 1205. geometric background and plane suppression techniques may be employed to filter out unwanted background objects, surfaces, or planar noise to better isolate or target the desired region of the tire. Thereafter, in a sixth step 1206, the tire axis is estimated using principal component analysis (PCA) or variance techniques. Cylindrical cropping can then be performed through axial and radial filtering in a seventh step 1207.

[0183] Next, in an eighth step 1208, clustering (e.g., DBSCAN) can be performed to extract the dominant tire surface region, and in a ninth step 1209, the largest tire cluster is selected. In a tenth step 1210, the tire cluster is normalized to the canonical orientation, e.g., using PCA-aligned axes. In this way, the filtered point cloud is normalized in scale and centered on an estimated tire midline to promote invariance across scanning conditions.

[0184] In doing so, the cluster is then downsampled while preserving tread features in an eleventh step 1211. For machine-learning purposes, point clouds may be downsampled to a fixed number of points (e.g., 2048 or 4096). Downsampling methods may include random uniform sampling, voxel-grid sampling, farthest-point sampling, or hybrid heuristics such as slice-based or curvature-weighted strategies. Advantageously, downsampling produces a consistent-size tensor suitable for batch processing in neural networks. The RGB values remain associated with each point after downsampling. Image data may undergo similar normalization, such as resizing to a canonical resolution (e.g., 224x224) and pixel-value normalization.

[0185] In a final step 1212, the point clouds are padded as needed to provide uniformly sized batches, and validity masks are applied to ensure the model ignores the padded points during training.

[0186] FIG. 15 depicts a multi-modal machine learning architecture 1300 according to an exemplary embodiment of the present disclosure. In one or more embodiments, the multimodal machine learning model 1300 receives both 3D point-cloud data and 2D image data. In one embodiment, the model includes two separate encoder networks, a first encoder 1301 for the point cloud data and a second encoder 1302 for the imagery data.

[0187] In one or more embodiments, the point cloud data encoder 1301 is based on architectures such as PointNet or its variants. The point cloud data encoder 1301 accepts a tensor 1303 containing XYZ and RGB values for each sampled point and applies multi-layer 4160065464perceptron (MLP) and / or 1D convolution layers 1304, batch normalization 1305, nonlinear activation functions, and global feature pooling (e.g., max pooling) 1306. The output is a point-cloud embedding vector 1307 representing geometric tread structure. In some embodiments, the RGB data may be omitted from input tensor 1303.

[0188] In one or more embodiments, the imagery data encoder 1302 receives the image input 1308 and applies any suitable convolutional or transformer-based neural backbone 1309, including pretrained networks such as ResNet-18, EfficientNet, or Vision Transformers, amongst other possibilities. In one or more embodiments, multiple frames may optionally be averaged 1310 to form a single representative embedding. In this way, the second encoder 1302 produces an image embedding vector 1311 capturing visual information from the tire tread such as texture, groove sharpness, debris or foreign material, wear indicators, and sidewall markings, amongst other possibilities.

[0189] In one or more embodiments, the system fuses 1312 the point-cloud and image embeddings. Fusion may be accomplished through concatenation, attention mechanisms, or gating or weighted fusion, for example. The fused representation is passed through fully connected layers 1313 to generate at least one of tread depth measurements 1314, wear pattern classifications 1315, defect or anomaly detections, sidewall condition indicators, and confidence values.

[0190] In addition to generating raw tread depth values, one or more embodiments of the presently disclosed system also compute higher-level analytics. For example, the system may compute wear rate over mileage, predicted replacement mileage, cost per mile metrics, tire position-based degradation comparisons, fleet-level tire performance benchmarking, and retread eligibility assessments, amongst other possibilities.

[0191] These outputs may be pushed to, or pulled from, an API of the system described herein and exchanged with fleet maintenance systems, tire dealer customer relationship (CRM) software, or inspection platforms. External systems may likewise push data to, or pull data from, the API to provide additional information used to enrich the outputs of the system, including installation dates, work orders, rotation history, retread cycles, mileage logs, driver information, vehicle information, timestamps, and location data.4260065464

[0192] FIG. 16 depicts an end-to-end processing pipeline 1400 for the backend system 170. As discussed, the captured point cloud and image data are packaged in a ZIP dataset 1401 and uploaded to the backend system 170. There, the images and point clouds are extracted 1402 and the binary formats are parsed 1403. As discussed above, the point cloud and image data are processed through filtering, tire surface isolation, and canonical data preparation 1404 as discussed above in relation to FIG. 14. Once the data is prepared, the multi-modal machine learning model inference can be applied 1405 to generate measurements 1406, such as tread depth and other qualitative metrics. The results are stored 1407 and used to compute various analytics 1408, such as wear rate, cost-per-mile, replacement predictions, etc. An API access layer is generated to provide secure external access 1409. In this way, fleet systems, dealer platforms, telematics, and applications can access the model 1410.

[0193] FIG. 18 schematically represents the contents of a supervised dataset 1600. As mentioned above, the supervised dataset 1600 is in an NPZ or other structured format. The dataset 1600 includes a point cloud tensor 1601 having shape N x 6 with values [X, Y, Z, R, G, B], wherein the RGB values are normalized to [0, 1] as described above. The structured dataset 1600 further includes an image tensor 1602 having the shape of H x W x 3 with RGB values normalized to [0, 1], Additionally, the supervised dataset 1600 includes measurement labels 1603 for machine learning model training. Such labels may include tread depth (in mm or 32nds of an inch), wear patterns, and tire conditions. Finally, the supervised dataset 1600 includes metadata 1604, such as tire position, vehicle ID and type, timestamp, scan filename, tire make, tire model, tire size, and any known tire issues. Such metadata may be input by the user 110 or captured through the scan using the user device 105.

[0194] FIG. 17 depicts a flow diagram 1500 comparing a tread scan and a sidewall scan according to one or more embodiments of the present disclosure. Both processes begin with a user device scan 1501 using, e.g., the LiDAR and camera of the user device 105. The tread depth and pattern scan has been described above and generally proceeds through the steps of multi-frame depth capture 1502, 3D reconstruction of the tread surface 1503, tread profile extraction 1504, and finally tread measurement output 1505, e.g.. as described in relation to FIG. 8A. In one or more embodiments, the user device scan 1501 may instead acquire a sidewall images or point clouds 1506. From this captured data, text or optical character recognition (OCR) may be utilized to extract information 1507 such as tire brand and model,4360065464DOT serial number 1508, load and speed ratings, visible sidewall defects (bulges, cracks, cuts, etc.) 1509, and pressure-related deformation at ground contact. Such defects in the sidewall may be classified 1510 and output to the user 110 through the app 106. These observations may be processed via computer-vision, OCR, or ML models. This information may enhance accuracy, enrich metadata, or enable automated record keeping in maintenance workflows.

[0195] FIG. 19 depicts a summary of the vehicle inspection machine learning flow 1700 based on the foregoing description. The process flow 1700 involves the user device 105 and the backend system 170. In particular, the process flow 1700 begins with user scanning of the tire through the app 106 in step 1701. The point cloud data (e.g., captured using LiDAR) and image data (e.g., captured using the camera) are sent to cloud storage in the ZIP archive in step 1702, and the labeled data as well as metadata entered by the user or captured through the app is sent to the datastore 1703. At the backend system 170, the dataset is retrieved and parsed in step 1704. For the image data, this involves decoding, resizing and normalizing as shown in a first branch 1705, and for the point cloud data, this involves point cloud parsing, filtering and tire isolation, canonical alignment, and downsampling and padding as shown in second branch 1706. The parsed data is assembled into a supervised dataset as shown in step 1707, and the machine learning model is applied to the dataset in step 1708. In the final step 1709, the system outputs a predicted tread depth and other tire or vehicle component quality metrics.

[0196] In performance of the foregoing processes, several factors may be considered when computing, analyzing, and tracking the tire measurements, tire wear patterns, tire wear rates, tire-related metrics, maintenance recommendations, and other related aspects. For example, one factor is fleet information including haul types, number of vehicles 125, number of drivers 110, driver 110 IDs and driver 110 behavior profiles, number of maintenance technicians 110, operating regions, common routes, number of orders, fleet asset characteristics, fleet preferences, daily mileage, weekly mileage, monthly mileage, maintenance schedules, etc. Other factors include vehicle 125, tractor 130, and trailer 140 mileages; vehicle 125. tractor 130, and trailer 140 routes driven; road surfaces of routes driven; road compound of routes driven; route characteristics of routes driven; weather during trips; speed driven during trips; load weight; load distribution; number of tires; make and model of tires; tire material composition; age of tires; tire rotation history; maintenance 4460065464schedules; maintenance crew and crew performance; and driver data (such as driver safety records, driver maintenance records, and driver ELD records), amongst other possibilities.

[0197] Users 110 of the app 106 may be granted maintenance rewards for scanning vehicle 125 components like tires 600 and labeling the component scan data with accurate and highly -detailed information such as manual tire measurements 620 taken by the user 110 using a tire gauge. The labeled scan data is used to train, refine, and validate machine learning models and further improve the accuracy of the system 100. Users 110 that are drivers may also be rewarded for having lower than average tire wear rates or vehicle issue rates. The rewards may come in the form of a digital currency or token, which may include one or more cryptocurrencies. The users 110 may also create or be provided with a digital currency wallet, i.e., cryptocurrency wallet, for storing these rewards. The users 110 may be rewarded based on various factors, including, for example, number of tires scanned in a given period, number of labeled tires scanned in a given period, quality of tires scanned in a given period, number of vehicle components scanned in a given period, number of labeled vehicle components scanned in a given period, number of tire maintenance issues found, number of tire maintenance issues resolved or mitigated, number of DOT compliance issues found and resolved, tire wear rates as compared to the fleet or other app users, or vehicle issue rates as compared to the fleet or other app users.

[0198] The fleets, companies, or agencies participating in a maintenance incentives program provide the rewards to the users 110. Companies and groups participating in such a program may include the app 106 parent company, trucking fleets, tire manufacturing companies, insurance companies, safety companies, US government departments and agencies such as DOT, FMCSA, DMV, and so forth.

[0199] Using embodiments of the presently disclosed system 100, on-device. Al-driven, and algorithmic analysis of vehicle maintenance data, including tire tread scan data, allow immediate results without reliance on backend servers. This capability ensures rapid and offline functionality, offering significant improvement over systems requiring cloud dependency.

[0200] Further, embodiments of the present disclosure provide business use cases and cost savings. For large fleets, which may log ~1 billion miles or more per year, saving even $0.01 per mile in tire costs, avoidable maintenance costs, or vehicle downtime, for example,4560065464results in millions of dollars in savings annually. Often, fleets set budgets by analyzing past expenditures and can leverage this system to optimize costs further, achieving savings beyond typical estimates.

[0201] All references, including publications, patent applications, and patents cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

[0202] The use of the terms “a” and “an” and “the” and similar referents in the context of describing the invention (especially in the context of the following claims) is to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly- contradicted by context. The terms “comprising,” “having.” “including,” and “containing” are to be construed as open-ended terms (i.e., meaning “including, but not limited to,”) unless otherwise noted. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.

[0203] Preferred embodiments of this invention are described herein, including the best mode known to the inventors for carrying out the invention. Variations of those preferred embodiments may become apparent to those of ordinary skill in the art upon reading the foregoing description. The inventors expect skilled artisans to employ such variations as appropriate, and the inventors intend for the invention to be practiced otherwise than as specifically described herein. Accordingly, this invention includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible4660065464variations thereof is encompassed by the invention unless otherwise indicated herein or otherwise clearly contradicted by context.4760065464

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method for inspecting a vehicle component, comprising:capturing, using a user device, three-dimensional spatial data of the vehicle component using a depth-sensing subsystem;capturing, using the user device, two-dimensional image data of the vehicle component using an imaging subsystem;generating a fused representation of the vehicle component by spatially aligning the three-dimensional spatial data with the two-dimensional image data;extracting geometric features from the three-dimensional spatial data and visual features from the two-dimensional image data;providing the extracted geometric features and visual features as inputs to one or more trained machine-learning models;deriving, using the one or more trained machine-learning models, a physical measurement or condition metric of the vehicle component based on the fused representation; andgenerating an inspection result indicating whether the physical measurement or condition metric satisfies a regulatory or maintenance criterion.

2. The method of claim 1, wherein the one or more trained machine-learning models comprise a multimodal model jointly processing the geometric features and the visual features.

3. The method of claim 1, wherein the one or more trained machine-learning models comprise a first model processing the geometric features and a second model processing the visual features, and wherein outputs of the first model and the second model are fused to derive the physical measurement or condition metric.48600654644. The method of claim 1. wherein the one or more trained machine-learning models are trained using labeled or pseudo-labeled inspection data collected from prior vehicle component scans.

5. The method of claim 1, further comprising generating a confidence score associated with the derived physical measurement or condition metric.

6. The method of claim 1, wherein deriving the physical measurement comprises computing a dimensional distance between surface features identified in the fused representation.

7. The method of claim 6, wherein the dimensional distance is computed between a first surface corresponding to a tread surface and a second surface corresponding to a groove base of a tire.

8. The method of claim 6, wherein the dimensional distance is computed using a plurality of sample points distributed across a surface of the vehicle component.

9. The method of claim 8, wherein a plurality of dimensional distances are aggregated to produce a representative measurement value.

10. The method of claim 1, wherein the vehicle component comprises a tire.

11. The method of claim 1, wherein the vehicle component comprises at least one of a brake component, axle, suspension component, lighting component, wheel assembly, or structural component.496006546412. The method of claim 1. wherein a plurality of vehicle components associated with a single vehicle are inspected during a same inspection session.

13. The method of claim 1, further comprising storing the fused representation and the inspection result in association with one or more identifiers enabling traceability, versioning, or longitudinal analysis of the vehicle component across multiple inspections.

14. The method of claim 13, wherein the one or more identifiers include at least one of a vehicle identifier, a component identifier, an inspection session identifier, a scan identifier, an operator identifier, a device identifier, a fleet identifier, or a model version identifier.

15. The method of claim 13, further comprising retrieving a prior fused representation corresponding to the same vehicle component captured at an earlier time.

16. The method of claim 15, further comprising spatially aligning the fused representation with the prior fused representation.

17. The method of claim 16, further comprising computing a change metric representing wear, degradation, or deformation of the vehicle component over time.

18. The method of claim 13, further comprising generating a historical report indicating changes in the physical measurement or condition metric across multiple inspections.

19. The method of claim 1, further comprising generating an alert when the inspection result indicates non-compliance with the regulatory or maintenance criterion.506006546420. The method of claim 19, wherein the alert is transmitted to a fleet management system associated with the vehicle.

21. The method of claim 1, further comprising providing inspection results via an application programming interface.

22. The method of claim 1, wherein the regulatory or maintenance criterion corresponds to a Department of Transportation out-of-service standard.

23. The method of claim 1, wherein the regulatory or maintenance criterion is configurable based on jurisdiction, vehicle type, or fleet policy.

24. A system for inspecting a vehicle component, comprising:one or more user devices including a depth-sensing subsystem and an imaging subsystem;one or more processors; andmemory storing instructions that, when executed by the one or more processors, cause the system to perform the method of any of claims 1-23.

25. The system of claim 24, wherein the one or more user devices comprise a mobile device or an extended-reality device.

26. The system of claim 24, wherein the system is configured to perform at least a portion of the method on the user device without reliance on a remote backend system.

27. The system of claim 24, wherein the system further comprises a cloud-based backend configured to aggregate inspection results from a plurality of vehicles.516006546428. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause performance of the method of any of claims 1 -23.5260065464

Images