Engine inspection system and method

Abstract

An aircraft component inspection system is provided. The system includes an image capture system including an image sensor system, a positioning system, and a processor configured to determine an inspection recipe based at least on an identifier associated with a component of an aircraft being inspected, identify a plurality of positions for image capture during an inspection workflow based on the inspection recipe, and determine a plurality of positions for image capture during the inspection workflow based on the plurality of positions. Machine instructions are provided to a positioning system to position an image sensor system relative to a component based on a plurality of positions, cause the image sensor system to capture images at the plurality of positions, and store the images with captured position data in an inspection data database.

Description

[0001] This application claims priority to Indian Patent Application No. 202311040040 filed on June 12, 2023, with the Indian Patent Office. The entire contents of the above application are incorporated herein by reference for all purposes. TECHNICAL FIELD

[0002] The teachings generally relate to inspection systems, and more specifically, to systems and methods for inspecting aircraft components, including jet engines. BACKGROUND

[0003] Aircraft engines undergo maintenance, repair, and overhaul (MRO) when sent to an original equipment manufacturer (OEM) or a partner MRO provider. During the initial stages of MRO, a shop can document the internal and external condition of the received engine. Typically, shop resources can take photos or videos of the exterior using available dumb cameras, which can result in inconsistent image capture, varying from engine to shop. It can take a significant amount of time to capture these frames and document them manually to keep records and communicate with customers when missing parts or other issues are found during inspection.

[0004] MRO of units, such as engines, can rely heavily on human visual inspection of the inbound unit to assess the condition of the unit, plan the maintenance work scope, and assess the quality of images or other data. Unconstrained, human-controlled data collection to support this assessment can result in inconsistent interpretation or produce insufficient and incomplete digital records, making condition assessment more difficult and inefficient. Tools, such as menu-driven inspections (MDI), can help improve consistency of data, but still produce significant differences in the quality and coverage needed to optimize the shop experience. It can also be difficult to obtain a complete digital record of the inbound unit when a human is the intermediary controlling the collection and interpretation of data to assess the condition. Many assessment decisions can be made at the time of inspection and without access to the complete digital record. Any subsequent issues can involve re-inspection of the unit due to the lack of a complete digital record. BRIEF DESCRIPTION OF DRAWINGS

[0005] The various needs are at least partially met through provision of the systems and methods for inspection guidance and data capture described in the following detailed description, particularly when studied in conjunction with the drawings. The detailed description includes specific details for the purpose of providing a thorough understanding of the aspects of the specification. However, it will be apparent to those skilled in the art that the aspects of the specification can be practiced without these specific details. In some instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the specification.

[0006] Figure 1 A block diagram of an inspection system including, in accordance with various embodiments.

[0007] Figure 2Flowchart including an inspection method according to various embodiments.

[0008] Figure 3 Flowchart including an inspection method according to various embodiments.

[0009] Figure 4 Flowchart including a model training process according to various embodiments.

[0010] Figure 5 Flowchart including a condition evaluation method according to various embodiments.

[0011] Figure 6 Flowchart including an inspection method according to various embodiments.

[0012] Figure 7A , Figure 7B , Figure 7C , Figure 7D , Figure 7E , Figure 7F , Figure 7G and Figure 7H Schematic diagram including an exemplary positioning system according to various embodiments.

[0013] Figure 8 Exemplary image capture system including according to various embodiments.

[0014] Figure 9 Flowchart including an inspection method according to various embodiments.

[0015] Figure 10 Flowchart including an inspection method according to various embodiments.

[0016] Figure 11A , Figure 11B and Figure 11C Schematic diagram including an image capture system and images acquired therefrom according to some embodiments.

[0017] Figure 12 Flowchart including an inspection method according to various embodiments.

[0018] Figure 13 Flowchart including an inspection method according to various embodiments.

[0019] Figure 14 Schematic diagram including features of an exemplary inspection user interface according to various embodiments.

[0020] Figure 15A , Figure 15B and Figure 15C Exemplary inspection user interface according to various embodiments.

[0021] Figure 16Example user input devices including according to some embodiments.

[0022] Figure 17 Example user input devices including according to some embodiments.

[0023] Figure 18 Example computer systems for inspection systems including according to various embodiments.

[0024] Elements of the drawings are shown exaggerated and not necessarily to scale for purposes of simplicity and clarity. For example, the dimensions and / or relative positions of some of the elements in the drawings can be exaggerated relative to other elements to help improve the understanding of various embodiments of the present teachings. Also, common but well-understood elements that are useful in a commercially feasible embodiment are often not depicted in order to facilitate a less obstructed view of these various embodiments. Certain actions and / or steps may be described or depicted in a particular order or sequence, but it is not necessarily meant to imply that there is an order or sequence other than the order or sequence implicitly revealed by the assertions of this patent or otherwise required by reference herein. It is therefore intended that the natural relationships existing, captured, and described in connection with the examples presented herein be understood that there can be anticipation, modification, and / or further input to the assistance of the process or method in its desired process. DETAILED DESCRIPTION

[0025] Reference will now be made in detail to embodiments of the present disclosure, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the present disclosure and not as a limitation thereto. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the present disclosure without departing from the scope or spirit of the present disclosure. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield still a further embodiment. Thus, it is intended that the present disclosure cover such modifications and variations as come within the scope of the appended claims and their equivalents.

[0026] As used herein, the terms "first," "second," and "third" can be used interchangeably to distinguish one component from another and are not intended to signify location or importance of the individual components.

[0027] Unless stated otherwise, as used herein the terms "coupled," "fixed," "attached to," "integrated," or the like, mean either directly coupled, fixed, or attached by one or more intermediate components or features.

[0028] The singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise.

[0029] In some aspects, the inspection systems and methods disclosed herein standardize engine configurations and conditions, collection, analysis, reporting, and storage of data regarding engine configurations and conditions. The inspection systems can include one or more application programming interfaces (APIs) that can be used on mobile devices. The inspection systems and methods disclosed herein can be applied to initial inspections for engine maintenance, repair, and overhaul / operation (MRO). Engine MRO can include all steps and actions involved in returning an engine to its working condition. Maintenance can include preventive, corrective, and predictive maintenance of an engine.

[0030] In some aspects, the inspection systems include one or more tools for capturing images or other data on engine hardware using standardized templates to guide operators (using visual overlays, reference images, or artificial intelligence (AI) guidance) or automation (robots) to provide documentation of engine configurations. The documentation can cover conditions of the entire exterior of the engine (or other ranges, such as modules, components, etc.). The tools can include robotic inspection systems. Operation of the robotic inspection systems can be directed and supervised remotely, enabling proper expert review and unattended local operation.

[0031] Manual guided “recipes” or automated investigation procedures of the inspection systems can be based on engine type or electronic serial number (ESN). The recipes can be shared between multiple devices and can include a mix of manual and automated functions where appropriate. The recipes can have deep link functionality to engine shop manuals (ESMs) or aircraft maintenance manuals (AMMs) to provide inspection requirements, limitations, part numbers and / or descriptions, assembly and / or disassembly instructions, torque values, etc. The recipes and / or templates can also be defined for ad-hoc purposes, e.g., by a user or a factory.

[0032] The tools of the inspection systems can include analysis to check for image exposure, blur, shake, etc., to ensure that the captured images meet the intent.

[0033] The tools of the inspection systems can enable capturing of part number markings, serial numbers, or other data plate entries. In one approach, the inspection systems can capture images of data plates associated with an engine or components thereof, and the inspection systems can extract information from the data plates using optical character recognition.

[0034] The images captured by the inspection system can then be used to determine the disposition of the hardware. The images can be uploaded to an online portal and used to generate a report documenting the findings of the investigation. The images, analysis, and / or disposition can be used to set and / or adjust the engine operating envelope. In some examples, the operating envelope can be adjusted in real-time based on the images, analysis, and / or disposition determined by the inspection system. For example, if damage details are found on a component, that finding can drive a component-specific workflow. The inspection system can be connected to a logistics infrastructure to assist in ordering of engine repair materials, required tools, and materials for engine test purposes (e.g., missing accessories), or for scheduling of labor or repairs or testing.

[0035] A user interface associated with the inspection system (e.g., an inspection portal) can allow inspectors and end users (e.g., customers, support functions, logistics, finance, engineering, etc.) to access data captured during the inspection. In some approaches, the presentation and display of data on the user interface can be customized for specific end users. The data can also be accessed via a web portal, a dedicated terminal, or other means. Data storage can be local (e.g., at the factory) and / or cloud-based (e.g., via a data lake, a cloud computing service such as Amazon Web Services (AWS), or LDP media). Further data acquired or generated by the inspection system can be integrated into an enterprise resource planning (ERP) system (e.g., SAP). Data communication of the inspection system can be wireless (e.g., Wi-Fi, Bluetooth) and / or wired.

[0036] In some aspects, the processes disclosed herein can improve end-to-end visibility of the inspection process, as well as the disposition of the engine throughout factory visits and on the wing. The inspection system can be used by customers to document engine conditions on and near the wing (e.g., prior to shipment), by logistics partners (e.g., at loading, unloading, delivery), and / or by the factory (e.g., engine manufacturers, suppliers, and third parties involved at various inspection stages and / or locations).

[0037] Images acquired by the inspection system can be tagged with metadata based on templates and / or recipes, and can be searchable through a portal (LDP media). Thus, engine, module, and component identification can be used to recall all images containing a search term to check conditions at various touchpoints and at various inspection stages.

[0038] In some aspects, provided herein is an inspection system that enables standardized processing of collecting, processing, and reviewing inspection data related to MRO activities. The inspection system can include a central computer having modules related to one or more of the following to simplify and integrate MRO activities: inspection recipe determination, image validation, part identification, part condition assessment, work scope planning, inspection report / documentation, and automation control. The inspection system can also include an inspection user interface (UI) through which internal and external inspections of an engine can be managed, analyzed, and used. The UI can provide multiple levels of data classification for analysis and report generation, e.g., at asset level, part level, customer level, region level, etc. The UI can be customized for various user groups, including MRO work scope, logistics, customers, suppliers / vendors, or enterprise resource planning (ERP) groups.

[0039] In some aspects, the inspection system can determine an inspection recipe that instructs an image capture device. The instructions can be automatic, manual, or a combination thereof. The inspection system can also validate and analyze images, and determine further MRO tasks based on the image validation and analysis. The inspection system can generate a report containing the inspection data, and initiate a logistics task, MRO instruction, or report based on the inspection data. Currently, non-standardized inspection processes result in different inspections for different service locations and product lines, which further results in non-standardized customer output files. In some embodiments, the systems and methods disclosed herein provide an inspection software application that provides standardization and guidance of part inspection processes. In some embodiments, the systems and methods disclosed herein can also be used for out-of-gate inspection of parts that have completed MRO. In some embodiments, the software application includes an artificial intelligence (AI)-assisted part layout determination. In some embodiments, the systems and methods disclosed herein can also improve the inspection quality of borescope inspections, and reduce the skill requirements of inspection operators.

[0040] In some embodiments, the systems and methods disclosed herein use defined templates and overlays of guidance to standardize engine exterior captures. The customizable templates can be shared across factories to standardize inspections across the network. Part layout determination within the templates further reduces errors in part registration. In some embodiments, the system provides a data pipeline to allow data to be uploaded and processed to relevant factory tools, avoiding additional manual data entry. The system can also provide automated notifications to downstream users (e.g., quality and engineering). In some embodiments, time-sensitive notifications can be generated at the time of inspection to alert customers, quality teams, and / or engineering teams of issues as early as possible. In some embodiments, post-inspection reports can be provided to assist customers in discussions and increase confidence in the status of the report. In some embodiments, the system also provides fully searchable metadata in images to improve the efficiency of part data searches.

[0041] In some embodiments, the inspection system can utilize AI and robotics to improve image consistency between inspections and enable predictive workscope application. In some embodiments, the process can reduce turn-around time and prevent engine re-ingestion (e.g., return for further inspection, repair, or overhaul). In some embodiments, the systems and methods disclosed herein reduce man-hours, distance traveled by workers, and enable real-time problem solving.

[0042] Reference is made to Figure 1 FIG. 1 shows an example aircraft component inspection system 100. In Figure 1 The inspection system 100 includes an inspection computer system 110 communicatively coupled to an image capture system 120, and a plurality of databases 130-138.

[0043] The inspection computer system 110 can include a processor-based device including one or more processors and memory. The inspection computer system 110 includes control circuitry, memory, and a network interface device for communicating with the image capture system 120 and / or the plurality of databases (e.g., databases 130-138). In some embodiments, the inspection computer system 110 can include one or more of control circuitry, microprocessors, central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), etc., and can be structured to execute computer-readable instructions stored on a computer-readable storage memory. The computer-readable storage memory can include volatile and / or non-volatile memory, and have computer-readable code stored thereon that, when executed by the processor, causes the inspection computer system 110 to perform inspection recipe determination, image validation, component identification, component condition assessment, workscope planning, inspection reporting and documentation, automated control, and / or machine learning model training to support aircraft component inspection operations. For example, further details of functions that can be performed by the inspection computer system 110 in accordance with some embodiments are described herein with reference to Figures 2-6 , Figures 9-10 and Figures 12-14 The inspection computer system 110 can be located locally or remotely from the image capture system 120, and communicate with the image capture system 120 via a wired connection, a wireless local area network connection, and / or over a wide area network. In some embodiments, the inspection computer system 110 can communicate with a plurality of image capture systems 120 in one or more inspection spaces to inspect one or more aircraft components.

[0044] The image capture system 120 can include automated image capture devices 122 and / or user-operated image capture devices 126. In some embodiments, the image capture system 120 can include multiple automated image capture devices 122 and / or multiple user-operated image capture devices 126.

[0045] The automated image capture devices 122 include a positioning system 123 and a sensor system 124. The positioning system 123 generally includes a mechanical system that is structured to change the relative positioning of the sensor system 124 and the component or part of the component being inspected. In some embodiments, the positioning system 123 can be physically coupled to one or more sensors of the sensor system 124 to move the sensors around the component. In some embodiments, the positioning system 123 is also structured to control the orientation of the sensors. In some embodiments, the positioning system 123 can be structured to manipulate the orientation of the component or part of the component to provide different views of the component to the sensor system 124. In some embodiments, the positioning system 123 can include a ground or aerial automated guided vehicle (AGV). In some embodiments, the positioning system 123 can include a snake-arm robot structured to perform endoscopic inspection of the interior of the component. For example, reference is made herein to Figure 6 、 Figure 7A - H and Figure 8 Examples of positioning systems that can be used with the inspection computer system 110 according to some embodiments are described.

[0046] The sensor system 124 includes one or more image sensors structured to capture images from the component being inspected. In some embodiments, the sensor system 124 can include multiple sensors of different sensor types (e.g., optical sensors, three-dimensional (3D) scanners, stereo cameras, infrared sensors, terahertz spectrometers, microwave imaging sensors, x-ray imagers, computed tomography scanners, eddy current imaging sensors, ultrasound imagers, etc.). In some embodiments, the sensor system can include a sensor array having multiple sensors having different focal lengths and / or orientations. In some embodiments, the inspection computer system 110 can be communicatively coupled to other types of sensors, such as gas sensors, acoustic sensors, thermal sensors, etc. For example, reference is made herein to Figure 6 and Figure 8 Further examples of sensor systems that can be used with the inspection computer system 110 according to some embodiments are described.

[0047] As Figure 1As shown, the user-operated image capture device 126 includes a user interface 127 and a sensor system 128. In some embodiments, the user interface 127 is configured to enable a user to capture images and adjust image capture settings. The user interface 127 can include one or more buttons, sliders, etc. for adjusting image capture settings. The sensor system 128 includes one or more image sensors configured to capture images from a component under inspection. In some embodiments, the sensor system 124 can include multiple sensors of different sensor types (e.g., optical sensors, 3D scanners, stereo cameras, infrared sensors, terahertz spectrometers, microwave imaging sensors, x-ray imagers, computed tomography scanners, eddy current imaging sensors, ultrasound imagers, etc.). In some embodiments, the sensor system can include a sensor array having multiple sensors having different focal lengths and / or orientations.

[0048] The inspection computer system 110 can be coupled to multiple local, remote, and / or cloud databases to retrieve data for performing the various functions described herein and / or store generated data. In some embodiments, the data stored in the databases can include training data 138, machine learning models 136, asset databases 134, recipe databases 132, and inspection databases 130. The training data 138 can be used to train one or more machine learning models 136 used by the inspection computer system 110. In some embodiments, the machine learning models 136 can include the recipe machine learning models described, the part identification models 136A and condition models 136B described, and / or the trigger condition models 136C described. For example, further descriptions of the training and use of machine learning models in accordance with some embodiments are provided herein with reference to these figures. Figure 4 The inspection computer system 110 can be coupled to multiple local, remote, and / or cloud databases to retrieve data for performing the various functions described herein and / or store generated data. In some embodiments, the data stored in the databases can include training data 138, machine learning models 136, asset databases 134, recipe databases 132, and inspection databases 130. The training data 138 can be used to train one or more machine learning models 136 used by the inspection computer system 110. In some embodiments, the machine learning models 136 can include the recipe machine learning models described, the part identification models 136A and condition models 136B described, and / or the trigger condition models 136C described. For example, further descriptions of the training and use of machine learning models in accordance with some embodiments are provided herein with reference to these figures. Figure 5 The inspection computer system 110 can be coupled to multiple local, remote, and / or cloud databases to retrieve data for performing the various functions described herein and / or store generated data. In some embodiments, the data stored in the databases can include training data 138, machine learning models 136, asset databases 134, recipe databases 132, and inspection databases 130. The training data 138 can be used to train one or more machine learning models 136 used by the inspection computer system 110. In some embodiments, the machine learning models 136 can include the recipe machine learning models described, the part identification models 136A and condition models 136B described, and / or the trigger condition models 136C described. For example, further descriptions of the training and use of machine learning models in accordance with some embodiments are provided herein with reference to these figures. Figure 9 The inspection computer system 110 can be coupled to multiple local, remote, and / or cloud databases to retrieve data for performing the various functions described herein and / or store generated data. In some embodiments, the data stored in the databases can include training data 138, machine learning models 136, asset databases 134, recipe databases 132, and inspection databases 130. The training data 138 can be used to train one or more machine learning models 136 used by the inspection computer system 110. In some embodiments, the machine learning models 136 can include the recipe machine learning models described, the part identification models 136A and condition models 136B described, and / or the trigger condition models 136C described. For example, further descriptions of the training and use of machine learning models in accordance with some embodiments are provided herein with reference to these figures.

[0049] The asset database 134 stores asset data, such as data regarding a plurality of aircraft components and / or parts. As used herein, an asset can refer to any aircraft component, such as an engine and engine components. In some embodiments, the asset database 134 can store asset tracking data, associating component / part identifiers with component / part descriptions and sales information. In some embodiments, the asset database 134 can also store other contextual / historical information, such as installation data, customer information, manufacturing dates, service and inspection history, usage information, contextual information, etc. The information stored in the asset database 134 can be used by the inspection computer system 110 to construct inspection recipes, inspection tasks, MRO tasks, etc. In some embodiments, the information stored in the asset database 134 can also be used for inspection reports and documentation.

[0050] The recipe database 132 stores inspection recipes to be executed by the inspection computer system 110. In some embodiments, the recipe database 132 can store multiple recipes each associated with a different aircraft part and / or part model / product line. In some embodiments, a recipe can specify a required data set to be captured during an inspection task of a part. In some embodiments, the data set can include required images that can specify a capture location and / or a capture configuration. In some embodiments, the recipe can also specify a part to be captured in each required image. In some embodiments, the recipe can require an order of images to be captured. In some embodiments, the recipe can also include requirements for the overall inspection task. For example, the recipe can require that the captured images collectively cover at least a percentage (e.g., 99%, 90%) of the surface of the part. For example, further details of inspection recipes are described herein with reference to Figure 3 and Figure 4 In some embodiments, one or more recipes can be adaptive recipes used in response to a detected trigger condition. For example, further details of adaptive inspection are described herein with reference to Figures 9-10

[0051] The inspection database 130 stores inspection data recorded via the image capture system 120 and the inspection computer system 110. In some embodiments, the inspection data for an inspection task can include multiple images with metadata attached. In some embodiments, the metadata can include one or more of a part identifier, a location identifier, a capture configuration identifier, a condition identifier, and the like. Further examples and descriptions of image metadata according to some embodiments are described herein with reference to Figure 3 and Figure 5

[0052] The inspection computer system 110 can also be coupled to a user interface device 140 that serves as a review and control center for the inspection. In some embodiments, a graphical user interface (GUI) can be provided on the user interface device 140 for reviewing captured images and associated data. In some embodiments, the user interface device 140 can also provide a control user interface that can be operated to remotely control one or more devices of the image capture system 120. Further descriptions of inspection review and control systems according to some embodiments are described herein with reference to Figures 12-16

[0053] ​​​In some embodiments, the inspection computer system 110 can also be communicatively coupled to other systems based on the inspection data. For example, the inspection computer system 110 can determine an MRO task and communicate instructions to the MRO system 141 for execution. The MRO system 141 can include a user interface for displaying the MRO task and / or an automated system for automatically performing the MRO task in response to receiving the task / instructions from the inspection computer system 110. In some embodiments, the MRO system 141 can return MRO data to the inspection computer system 110 for storage as training data 138 for training of the machine learning model 136. In some embodiments, the inspection computer system 110 can also communicate with a logistics / purchasing system 142 to initiate ordering of missing or damaged parts identified during the inspection process.

[0054] In some embodiments, one or more of the inspection computer system 110, the image capture system 120, the user interface device 140, the logistics / purchasing system 142, and the MRO system 141 are implemented as Figure 18 the computer system 1810.

[0055] In Figure 18 the computer system 1810 includes a processor 1811, a memory 1812, an input / output (I / O) adapter 1813, and a network adapter 1814 that communicate over a bus. In some embodiments, the computer system 1810 can include other common components of a processor-based device. The processor 1811 is configured to execute computer-readable instructions stored in the memory 1812 to perform one or more of the functions described herein. The processor 1811 is also configured to receive and / or transmit data via the I / O adapter 1813 and / or the network adapter 1814. In some embodiments, the input device 1815 and the output device 1816 can include user interface devices such as a display screen, a touchscreen, a keyboard, a microphone, a speaker, a camera, a motion sensor, etc. In some embodiments, the computer system 1810 can communicate with one or more other devices or databases over a network 1817 via the network adapter 1814. In some embodiments, the network 1817 can include a local or wide-area network, such as the Internet. While the computer system 1810 is shown as having a single processor 1811 and memory 1812, in some embodiments, the computer system 1810 can be implemented on a cloud-based computer having multiple distributed processors and memories.

[0056] Referring next to Figure 2 a process of conducting an aircraft component inspection is shown. Figure 2 The steps in Figure 2 may be performed by one or more processor-based devices. In some embodiments, one or more of the steps of Figure 2 may be performed by one or more of the devices described herein with reference toFigure 1 The described inspection is performed on the computer system 110, the image capture system 120, and / or the user interface device 140.

[0057] In step 210, an inspection is initiated. In some embodiments, the inspection may be an initial inspection of the aircraft component at an MRO facility. The inspection task may provide documentation of the component's initial state prior to MRO processing. In some embodiments, the component may be cleaned (e.g., via foam cleaning) prior to step 210 to provide better visibility of the component's parts and surfaces. In some embodiments, the aircraft component may be a turbocharged engine. In some embodiments, the inspection system and processing described herein may also be used to inspect other aircraft components, such as fuselages, wings, landing gear, fuel systems, etc. In some embodiments, a component identifier may be provided in step 210. In some embodiments, the component identifier may be input by a user, provided by a mission management system, and / or captured by an imaging system.

[0058] In step 220, the system determines the inspection recipe for the inspection task and determines the initial instructions. In step 230, the system instructs the image capture system (e.g., based on the initial instructions) to... Figure 1 Image capture system 120). For example, this article refers to Figure 3 , Figure 4 and Figure 6 Further details of the inspection formula and initial instructions determined according to some embodiments are described.

[0059] In step 240, the system verifies the captured image received from the image capture system. If the image fails verification, the system can update the capture command in step 245 to cause the capture of a new image. In step 250, the system also evaluates the condition of the component and / or parts of the component based on the captured image. If a triggering condition is detected, the system can cause the capture of an additional image by updating the capture command in step 245. For example, this document references... Figures 3-6 , Figures 9-10 and Figure 12 Further details of image verification and condition evaluation according to some embodiments are described.

[0060] In step 260, the system appends metadata and stores the captured image in the inspection database 130. In some embodiments, the metadata may include image capture system location, image capture system orientation, image capture system identifier, imaging component location, component identifier, and / or timestamp. In some embodiments, the inspection recipe associates the desired image with a component identifier, and the metadata includes the component identifier from the inspection recipe. In some embodiments, the metadata includes a component identifier determined based on machine object detection, machine feature detection, object recognition algorithms, and / or optical character recognition algorithms. For example, this document references... Figure 1 and Figures 3-6 Further details regarding image metadata and component identification are described.

[0061] In step 270, the system can determine MRO tasks based on the inspection results. In step 275, the system can provide a user interface for reviewing the inspection results. The inspection output can be used to initiate logistics tasks in step 277, provide MRO instructions in step 278, and / or generate reports for internal or customer use in step 279. For example, this document references... Figures 12-16 Further details regarding the inspection review, control user interface, and inspection reports are described.

[0062] In some embodiments, the systems and methods described herein can provide complete engine inspection guidance and data capture. Inspection systems and methods may include documenting the condition of the entire assembled engine when it is received for servicing and when it is released back to the customer. This documentation documents the necessary scope of work to facilitate service access, to help verify the completion of the scope of work, and as a reference in any disputes regarding the condition of the engine. In some methods, inspection systems and methods may utilize photographic view recipes – from experts, manuals, simulations (such as computer-aided design (CAD)), etc. In some methods, inspection systems and methods may involve obtaining photographic guidance from the engine inspection recipe. In other methods, inspection systems and methods may involve verifying that the photographs match the recipe. In other methods, inspection systems and methods may involve verifying that the photographs provide complete or adequate 3D coverage of the engine. In other methods, inspection systems and methods may involve automatically tagging images with photographic content for downstream processing.

[0063] Conditioning the entire engine for documentation involves collecting a photo set that fully or sufficiently covers the exterior and interior components of the engine in sufficient detail (e.g., sufficient magnification, perspective, and resolution) to define the repair scope and verify completion of the scope. One challenge in collecting these photos is ensuring that the photos cover the entire engine and are of high quality. For example, it can be useful to ensure that the photos are taken from a viewing perspective that presents key components and surfaces in sufficient resolution, detail, and lighting to allow interpretation of the scope definition and verification. There can be additional challenges in establishing photo integrity for missing engine components or severely damaged components.

[0064] The inspection systems and methods described herein can guide the collection of the entire engine exterior and interior photos by: (1) directly leveraging or learning from expert knowledge and simulations (CAD) to define camera positions that will capture key components and surfaces from the appropriate perspective and in sufficient resolution; (2) guiding the user (e.g., via augmented reality) or automated systems through the sequential capture of these images by referencing images, visual overlays, and positioning cues (e.g., up / down pan, left / right pan, in / out pan; rotation by adjusting pitch, yaw, roll); and (3) verifying full or sufficient coverage by image comparison or assessing the integrity of 3D reconstruction. Additionally, the inspection systems and methods described herein can tag images with information for each particular engine component or surface (e.g., geotag images) for downstream use.

[0065] The inspection systems and methods can directly leverage experts to define recipes to capture high quality and complete photo sets. Additionally, the inspection systems and methods can indirectly leverage maintenance manuals (e.g., using AI) to define a portion of the component and surface views shown in the graphics in the manual. The inspection systems and methods can also indirectly leverage experts using AI to learn how experts capture high quality and complete photo sets. The inspection systems and methods can also directly leverage CAD (e.g., via product views) information and camera simulations using AI to define a minimum view set that meets component and surface requirements and integrity. The inspection systems and methods can also guide the collection of photos by referencing images, by screen overlays, and / or by screen navigation and positioning cues (e.g., left / right pan, up / down pan, in / out pan; pitch, yaw, roll rotation). The inspection systems and methods can also guide the collection of images by projecting targets (e.g., via lasers) onto components.

[0066] In some approaches, the inspection systems and methods can verify the integrity of the photo set via comparing the images to a reference set using AI. The inspection systems and methods can also verify the integrity of the photo set via 3D reconstruction and identify differences between the reconstruction and the expected reconstruction.

[0067] In some methods, the inspection systems and methods can also tag images directly with target content information such as engine serial number (ESN), date / time, component imaged, serial number, or other information. The systems and methods can also indirectly tag images with information such as ESN, date / time, component imaged, serial number, etc. using object detectors (e.g., via AI), feature detectors (e.g., via AI), and OCR (e.g., via AI).

[0068] Referring next to Figure 3 , a method for inspection guidance is provided. The method can be performed based on communications between an inspection computer system 110 and an image capture system 120. In some embodiments, the image capture system 120 can be a user-operated image capture device 126 operated by an inspection operator in an inspection area near an aircraft component being inspected, such as an engine. In some embodiments, the image capture system 120 can include automated image capture devices 122 including one or more image sensors and one or more automated positioning devices. The inspection computer system 110 can be located within or near the inspection area, at a remote location, or in a cloud network. In some embodiments, the inspection computer system 110 can be implemented as a plurality of physically separate processor-based devices. In some embodiments, the inspection computer system 110 can communicate inspection instructions to a plurality of image capture devices, such as a plurality of automated image capture devices 122 having different sensor types and / or a combination of automated image capture devices 122 and user-operated image capture devices 126.

[0069] In step 212, the system receives a component identifier. In some embodiments, the component identifier can be entered via a user interface device, received from an asset or task management system, and / or captured by an image capture device (e.g., the image capture system 120 of Figure 1 In some embodiments, the component identifier can include a component / engine serial number, a model identifier, a product line identifier, etc.

[0070] In step 220A, the inspection computer system 110 determines an inspection recipe. In some embodiments, the inspection recipe is retrieved from the recipe database 132 based on the identifier associated with the part. In some embodiments, the inspection recipe is generated based on a recipe machine learning model trained on a plurality of image sets captured from a plurality of parts, physical part models, and / or computer part models. The images in the training set can be labeled with image quality metrics that identify the image sharpness and data quality of the images. The images in the training set can include captured information used as training data for the machine learning model, such as capture location, capture orientation, sensor configuration, etc. The recipe machine learning model can be structured to determine capture location, orientation, and / or sensor configuration to increase / maximize the image quality metrics for the inspection. Referring to Figure 4 Further details of the recipe generated by the machine learning algorithm are described. In some embodiments, the inspection recipe is generated based on mapping reference images in a maintenance manual associated with the part with a computer model of the part using computer vision algorithms. In some embodiments, the inspection recipe is generated based on simulating camera views on the computer model of the part to define a minimum set of views that cover predefined portions of the part.

[0071] In some embodiments, prior to step 220A, the inspection computer system 110 also retrieves part context data, such as part inspection history, part repair history, customer specified task requirements, identified issues, part usage history (e.g., flight path heat map), and / or geographic region of the part being inspected or parts of the same make, model, customer, and usage history. In some embodiments, the inspection recipe is also determined based on the context data. For example, for parts with a particular usage history or geographic region, additional or modified required images and / or capture locations can be added to the recipe. In a specific example, an engine used by an airline whose routes are concentrated in desert regions, the recipe can include additional required images of areas of the engine that are particularly prone to sand and wear damage. In some embodiments, the context data can be retrieved from an asset database 134 that stores customer information, usage history, and repair history for various parts of the engine or aircraft. In some embodiments, the context data can also be included in the training data set used to train the recipe machine learning model.

[0072] In some embodiments, the inspection recipe identifies a plurality of required images to be captured during an inspection work scope. In some embodiments, the inspection recipe specifies an indicated image capture position associated with a required image, where the indicated capture position comprises a coordinate position relative to the inspection space, the part, or a feature of the part. In some embodiments, the inspection recipe specifies an indicated image capture position associated with a required image, where the indicated capture position comprises a distance to a feature of the part. In some embodiments, the inspection recipe specifies an indicated image capture orientation, including roll, pitch, and / or yaw of a sensor of the image capture system. In some embodiments, the inspection recipe specifies a view of the part for a required image, where the view of the part defines a size and / or orientation of a portion of the part within an image frame. In some embodiments, the inspection recipe specifies an image type and / or image capture system type for a required image. In some embodiments, the image capture system includes multiple types of sensors, such as optical sensors, infrared sensors, terahertz spectrometers, microwave imaging sensors, x-ray imagers, computed tomography scanners, eddy current imaging sensors, or ultrasonic imagers.

[0073] In step 220B, the inspection computer system 110 determines capture instructions for the user interface based on the recipe. In some embodiments, the inspection computer system 110 can determine instructions for each required image in the recipe. In some embodiments, the instructions can include one or more of a capture position, a capture angle, and a capture configuration.

[0074] In step 310, an inspection user interface is provided on the image capture system 120. In some embodiments, the inspection user interface can include a graphical user interface of an application executing on a mobile device, such as a tablet, a head-mounted display device, and / or an AR or VR device. In some embodiments, for automated image capture devices 122, step 310 can be omitted. In some embodiments, for automated image capture devices 122, an inspection and control user interface can be displayed on a user interface device while capture is in progress. An operator of the inspection and control user interface can inspect images captured by the automated image capture device 122 and adjust the capture position, capture configuration, and / or capture timing for additional / subsequent captures.

[0075] In step 320, the capture instructions are executed on the user interface. In some embodiments, the instructions include machine instructions for controlling movement of a positioning system of the automated image capture device 122. In some embodiments, the machine instructions can include positioning instructions for the positioning system 123 and capture configuration and timing information for the sensor system 124.

[0076] In some embodiments, the capture instructions are performed by displaying instructions on a user interface display of a user-operated image capture device 126. In some embodiments, the instructions include an augmented reality or mixed reality display displayed on the image capture system 120 that superimposes the instructions on a view of a portion of the component. In some embodiments, the instructions displayed on the user interface device include a reference image of the portion of the component to be captured, an outline of the portion of the component to be captured, and / or an image of the component with identifiers marking the locations of the portion of the component to be captured. In some embodiments, the system includes a projection display device, and instructions from the set of instructions are projected onto the surface of the component under inspection and / or around the component under inspection.

[0077] In some embodiments, the inspection computer system 110 and / or the image capture system 120 are configured to identify an image capture system position relative to the component based on a position sensor on the image capture system, images captured by the image capture system 120, or images captured by a separate sensor system, and provide capture instructions to the image capture system 120 based on the image capture system position. In some embodiments, the inspection computer system 110 and / or the image capture system 120 are further configured to determine an image capture system orientation relative to the component, and provide capture instructions further based on the image capture system orientation.

[0078] In some embodiments, the instructions transmitted from the inspection computer system 110 to the image capture system 120 are configured to automatically set image capture configurations on the image capture system, where the image capture configurations include zoom level, exposure time, light sensitivity settings, lighting settings, image resolution, video length, and / or sensor type selection. For example, the instructions can be configured to change settings of a mobile device camera via a camera API. In some embodiments, the instructions can be configured to change settings of sensor devices of an automated positioning system. In some embodiments, the image capture is triggered by an operator of the user-operated image capture device 126 via a graphical user interface in response to display of the capture instructions. In some embodiments, the inspection computer system 110 and / or the image capture system 120 can automatically trigger capture when a sensor and / or user device is detected to be in a capture position and / or orientation. In some embodiments, the image capture system 120 can continuously record images, and the inspection computer system 110 and / or the image capture system 120 can selectively transmit and / or store frames / images that match position and / or quality requirements of an inspection recipe.

[0079] In some embodiments, the inspection computer system 110 is also configured to provide instructions to the second image capture system based on the recipe to capture the required images specified in the recipe simultaneously. For example, two or more operators and / or automated image capture devices 122 can coordinate and capture images simultaneously during the inspection work range of the part.

[0080] In step 330, the image capture system 120 transmits the captured images to the inspection computer system 110. In some embodiments, the captured images can include still images or video having multiple frames and / or multiple channels. In some embodiments, the captured images are transmitted with metadata identifying the capture location and / or capture configuration recorded during capture of the images.

[0081] In step 240, the inspection computer system 110 validates the images received from the image capture system 120. In some embodiments, the images are validated based on image capture system location and / or image quality. In some embodiments, the images are validated based on recipe requirements for clarity (e.g., focus blur, motion blur, brightness, sharpness) and data quality. In some embodiments, the inspection recipe identifies a portion of the part associated with the required image, and the captured image is validated based on detecting the portion of the part in the captured image based on a machine object detection, a machine feature detection, an object recognition algorithm, and / or an optical character recognition algorithm. In some embodiments, the inspection recipe identifies a portion of the part associated with the required image, and the captured image is validated based on comparing the captured image to a computer model of the part, a previously captured image from the part, and / or a previously captured image from a similar part associated with the portion of the part. In some embodiments, the inspection recipe identifies a computer model of the part, and the captured image is validated based on identifying gaps in coverage by comparing the computer model and the captured image, and the processor is further configured to indicate additional capture tasks based on the gaps in coverage. In some embodiments, the inspection computer system 110 is configured to validate the inspection by comparing the portions of the part imaged by the images captured during the inspection work range to completeness requirements specified in the inspection recipe. In some embodiments, the image capture system 120 can be configured to perform some or all of the image validation described with reference to step 240.

[0082] If the image fails validation, the system returns to step 220B. For a user-operated image capture device 126, the operator can be instructed to re-capture the image. In some embodiments, the user instruction to re-capture can include capture suggestions (e.g., move closer to the area of interest, increase lighting, remain still, clean the lens, etc.). In some embodiments, the capture suggestions can be determined based on the image issues identified in step 240. For an automated image capture device 122, the inspection computer system 110 can determine modified machine capture instructions in step 220B. In some embodiments, the modified capture instructions can be determined based on a predetermined detail integration recipe, a predetermined variation capture recipe, and / or by analyzing the captured image. Reference is made herein to Figures 9-10 Further details of adapting image capture are described. The image capture system 120 can capture a new image based on the re-capture instructions, and the new image can be validated again at step 240. In some embodiments, steps 220B, 320, 330, and 240 can be repeated until an image is captured that meets the requirements of the inspection recipe. In some embodiments, the inspection recipe can include overall inspection task requirements, such as overall coverage, completeness, and amount of overlap between one or more images. The captured images can be collectively validated according to the overall requirements before the inspection task is completed.

[0083] If the image passes validation, in step 260A, the inspection computer system 110 appends metadata to the image for storage in the inspection database 130. In some embodiments, the metadata includes capture metadata recorded by the image capture system 120. In some embodiments, the metadata can include image capture system location, image capture system orientation, image capture system identifier, imaging component location, component identifier, and / or a timestamp. In some embodiments, the inspection recipe associates a required image with a part identifier, and the metadata includes the part identifier from the inspection recipe. In some embodiments, the metadata includes a part identifier determined based on machine object detection, machine feature detection, object recognition algorithms, and / or optical character recognition algorithms. Reference is made herein to Figure 5 Further details of part identification are described. In some embodiments, the image metadata can include capture configurations, such as zoom level, focal length, exposure time, light sensitivity setting, lighting setting, image resolution, video length, and / or sensor type selection. In some embodiments, the inspection computer system 110 can overlay at least a portion of the metadata (e.g., part name, component identifier, capture data) on the captured image.

[0084] In some embodiments, the inspection computer system 110 can also be configured to identify anomalies based on data recorded by the image capture system. In some embodiments, the anomalies can be appended as metadata. In some embodiments, the inspection computer system 110 can modify the instructions communicated to the image capture system 120 based on the anomalies prior to completing the inspection work scope. In some embodiments, the inspection computer system 110 can also determine a work scope for a repair or maintenance task based on the plurality of images captured by the image capture system 120 in response to receiving the instructions from the processor.

[0085] Referring next to Figure 4 , an example process of generating an inspection recipe via machine learning is shown. In some embodiments, Figure 4 the steps in may be performed by the inspection computer system 110 and / or a separate machine learning computer system.

[0086] In step 230, the system instructs image capture via image capture system 120. In step 235, the system receives captured images with capture data. In some embodiments, the capture data can include capture location and capture configuration. In step 242, the system can determine image sharpness metrics and data quality metrics. In some embodiments, step 242 can be automated by the system through image analysis algorithms, object detection algorithms, etc. In some embodiments, step 242 can include scores inputted by an inspector. The image capture data and metrics are stored as training data 138. In some embodiments, the images are also included in training data 138. In some embodiments, the images can be tagged with other data such as part identifier, component identifier, component context data, and / or factory data. In step 401, the system processes the training data to filter and organize information relevant to recipe machine learning model training. In step 402, the recipe model is trained / retrained with training data 138. In some embodiments, the machine learning model can be trained with multiple images tagged with capture location, capture configuration, and image metrics to select a set of capture locations to optimize image metrics for a recipe. In some embodiments, the machine learning algorithms described herein can be supervised, unsupervised, or reinforcement machine learning algorithms. In some embodiments, the machine learning algorithms can include decision trees, linear regression, neural networks, descriptive models, Q-learning, deep adversarial networks, and / or time-difference algorithms. In some embodiments, the multiple images can be tagged with other data from training data 138 including context data, and the recipe machine learning model can generate a recipe based on the context data. In some embodiments, the training data can include image data of multiple aircraft components of the same or different types captured by multiple image capture devices and multiple operators at multiple factory locations. In some embodiments, the training data can also include images from other sources such as computer model simulations, historical inspection images, and service or operations manual images.

[0087] In some embodiments, the recipe database 132 can store machine learning models associated with various aircraft components by type, model number, etc. In some embodiments, the recipes can also be associated with other context data such as customer information, geographic information, MRO history, etc. In some embodiments, the trained recipe models can be stored in recipe database 132, and component information received for initiating an inspection task is used as an input set for the recipe machine learning model to generate a recipe for the inspection task.

[0088] The inspection systems and methods described herein can use robotic control and digital instrumentation data acquisition to drive consistency and completeness of data collection for evaluating unit conditions for drive shop range planning processes. Portions of the inspection process can reduce incomplete or incorrect condition evaluations by digitally planning and directing inspection shop ranges to ensure complete information is collected. Directing and standardizing can eliminate or reduce human factors or fatigue that can lead to missed indications and incorrect condition evaluations. Consistency of data collection can then be leveraged to improve effectiveness of AI-assisted condition evaluations and establish complete digital records of inbound unit conditions. Various digital records can then be connected to MRO logistics in order to control aspects from labor optimization and equipment utilization to inventory management and customer turn-around time estimates.

[0089] In some embodiments, the inspection method for engine MRO can involve artificial intelligence / machine learning condition evaluation. The shop range planning for engine MRO can begin with unit inspection to evaluate conditions and determine any recommendations and required repairs that will enable the unit to return to service or meet specific mission capability requirements. The inspection method can include AI-assisted condition evaluation, such as AI-driven unit and / or part evaluation. AI-assisted condition evaluation can include determining the presence or absence of components and digital shop range planning, thereby improving factory utilization and reducing turn-around time.

[0090] The presence or absence of components or accessories can drive shop ranges and connect with logistics functions, such as equipment scheduling, selection of technician skill sets, and inventory management, which can help estimate and streamline MRO processes. The inspection method leverages AI and / or computer vision (CV) technology to enable identification of these components by location and other attributes of the disassembly requirements of the unit, such as color, shape, and geometry. These attributes can also be used to ensure the unit is returned to the customer with all appropriate accessories either cataloged with the inbound record or deemed suitable for exchange or replacement. Determining the appropriate accessories associated with the unit can involve using AI / OCR to identify data plate serial numbers. The inspection method can also involve connecting to ERP systems. These technologies can also be used to address part manufacturer approved (PMA) part and counterfeit identification issues, allowing identification of unapproved parts. Component identification and / or condition can also lead to different shop range planning procedures, such as additional borescope inspections (BSI), or can involve use of additional inspection modalities.

[0091] Depending on the customer or regional context, observations based on unit condition can have different interpretations, where the mission profile can impact the remaining life, value, and maintenance procedures of the asset. The inspection method leverages AI to assess condition and establish a digital record of unit and / or part condition. The digital record can allow for comparison to design and service limits and can quickly communicate the evidence required by the customer to understand whether the unit will meet mission requirements and / or obtain necessary repair authorization. A historical record of previous outcomes can serve as further evidence of the current unit maintenance justification. This inspection method can also allow for combining unit and / or part condition (indicative of type, location, size, aggregation within an assembly, etc.) with life and durability data and customer-specific data such as unit context (geographical routing, cycling data, etc.) to establish rules by which automatic or semi-automatic authorization can be obtained from the customer, while providing the customer with visibility into the repair workflow. AI-driven condition assessment can also be used to assign asset valuations for units ending lease or being sold for redeployment or scrapping.

[0092] Referring next to Figure 5 , a process for condition assessment with an inspection system is shown. In some embodiments, Figure 5 one or more steps in the process of FIG. 2 can be performed by a processor-based device, such as the inspection computer system 110. In step 235, the system receives captured images. In some embodiments, the images can be received in response to the indication of image capture in step 230 described with reference to Figure 2 In some embodiments, the images can be captured by the automated image capture device 122 and / or the user-operated image capture device 126. While image data is generally described herein, in some embodiments, condition assessment described in FIG. 2 can also be based on other types of data, such as gas emission data, acoustic data, electrical signals, airflow measurements, torque measurements, etc. Figure 5

[0093] ​In step 241, the system identifies one or more parts of the component based on the one or more images received in step 235. In some embodiments, a part identification model 136A is used to identify the parts. In some embodiments, the parts are identified based on identifying data placards (e.g., serial number placards, model identifier placards) associated with the multiple parts in one or more of the captured images. In some embodiments, the parts are identified based on optical character recognition algorithms performed on one or more of the captured images to identify identifiers (serial numbers, part numbers, model numbers, etc.). In some embodiments, the parts are identified based on part shapes, colors, and / or component locations appearing in one or more of the captured images. In some embodiments, the parts are identified using part identification machine learning models and / or computer vision algorithms. In some embodiments, the parts can be identified based on comparing the captured images to reference images of parts of the component. The reference images can be photographs, 3D models, and / or service / operation manual images. In some embodiments, the parts can be identified based at least in part on the position of the image sensor relative to the component. For example, the system can identify the parts of the component in the images based on the parts expected to be within the field of view of the sensor at the sensor’s position. In some embodiments, the system can simulate the position of the sensor with a 3D model of the component to determine the parts that can be within the field of view of the sensor.

[0094] In some embodiments, the system further validates the received images in step 240. In some embodiments, the clarity (e.g., blurriness, brightness, focus, sharpness) of the images is validated prior to step 241. In some embodiments, the parts identified in step 241 can be used to validate the data quality of the images. For example, the system can determine whether the images cover the areas of the component specified by the recipe based on the part identification in step 241. In some embodiments, if the image validation fails, a recapture can be indicated, e.g., Figures 2-3 In one example, for an aircraft engine, the parts that can be identified can include a housing surface, a mount, a switch, a valve, a connector, a seal, a bushing, a wire, a tube, a fastener, a spacer, a port, a turbine, a blade, a vane, an airfoil, a shaft, etc.

[0095] In step 250, the system identifies a condition of the identified part based on the captured image. In some embodiments, the condition of the part is determined based on detecting an anomaly in the image of the part, which can correspond to wear or damage of the part. In some embodiments, the condition of the part can be determined based on a size, shape, color, 3D profile, and / or location of the anomaly. In some embodiments, the condition can be determined based on other types of data such as gas emission data, acoustic data, electrical signals, airflow measurements, torque measurements, etc. In some embodiments, the condition of the part is identified using the part identification model 136A. In some embodiments, the condition of the part includes a presence or absence of the part, a serviceability of the part, a mission capability of the part, and / or a maintenance or repair mission for the part. In some embodiments, the system can compare the part to records in an asset database to determine authenticity, manufacturer, or origin of the part. In some embodiments, the system also compares the part to a set of manufacturing standards to determine standard compliance of the part. For example, the system can determine whether an aftermarket part meets original equipment manufacturer (OEM) standards. In some embodiments, the system can compare the condition of one or more parts to a mission requirement profile to determine a mission suitability matrix for the part. In some embodiments, the system also compares the condition of one or more parts to market value reference data to determine a market value of the component or one or more parts of the component. In some embodiments, in response to a condition of the part being absent or unusable, the system can be configured to forward a procurement request to a logistics system to initiate procurement of the missing or damaged part.

[0096] In step 252, the system determines and instructs a follow-up mission. In some embodiments, the follow-up mission includes a further inspection mission, a repair mission, or a maintenance mission. In some embodiments, the follow-up mission can include a detailed inspection mission instructed to the image capture system 120 to acquire additional images, and steps 235, 240, 241, and 250 can be repeated on the additional images. In some embodiments, the follow-up mission can be a reference Figure 9 to the described fitment inspection mission or MRO mission.

[0097] In step 280, the system receives feedback regarding the part identification, condition identification, and / or subsequent task determination. In some embodiments, the feedback can be provided by an inspector. In some embodiments, the feedback can include a success or failure of a previous inspection or MRO task. In some embodiments, the part identification, condition identification, subsequent task, and / or feedback can be stored as training data for training the part identification model 136A and / or the condition model 136B. In some embodiments, the training data 138 can include a plurality of images of parts labeled with a part identifier. In some embodiments, the part identification model 136A can be trained with a plurality of images of parts and a part identifier associated with each part. In some embodiments, the images in the training data set can include photographs, 3D part models, and / or MRO manual figures. In some embodiments, the part identification model 136A can be used with a machine vision algorithm to output a part identifier based on an input image. In some embodiments, the condition model 136B can be trained with a plurality of images of parts labeled with an associated condition. In some embodiments, the images can also be labeled with background information, such as part age, part usage history, part customer history, part flight path history, part service history, part repair history, etc. In some embodiments, the images can also be labeled with other associated sensor data, such as gas emission data, thermal distribution data, acoustic data, etc. The condition model 136B can be trained to receive one or more images of a part captured in step 235, and optionally other background and sensor data, and output a condition associated with the part. In some embodiments, the feedback received in step 280 can be used to select and / or filter data from the training data 138, and / or place weighting factors on data in the training data set.

[0098] In some embodiments, in step 410, the system processes the training data to filter and organize information related to part identification and condition assessment model training. In step 420, the part identification model 136A and / or the condition model 136B are trained / retrained with the training data 138. The system is structured to process / filter data in the training data 138 based on the trained models, and execute machine learning algorithms to build and update the part identification model 136A and the condition model 136B. In some embodiments, the part identification model 136A and the condition model 136B can be combined into a single model. For example, a combined model can be trained with images labeled with both a part identifier and a condition identifier. In some embodiments, the combined model can also be trained with images labeled with background data and other sensor data.

[0099] The inspection system and method described herein can also be combined with imaging devices and controls. The inspection system and method may include pre-configured sensors to record the condition of external engine parts / surfaces. The inspection system and method may also integrate multi-mode sensors to construct imaging devices for automatic parameter setting, thereby improving inspection quality. Additionally, the inspection system and method can utilize AI-assisted multi-sensor inspection to reconfigure sensors, detect and identify engine construction, identify anomalies, and digitize the complete inspection. Furthermore, the inspection system and method can utilize AI-assisted quality assurance in real time to ensure that appropriate quality is recorded before asset movement. AI-assisted quality assurance can address imaging errors that may arise due to factors such as lighting, movement, and focusing.

[0100] Imaging devices and controls can facilitate a consistent positioning system that can: 1) pre-construct sensors to document the engine's condition based on nominal CAD specifications; 2) integrate multi-mode sensors to construct imaging devices for setting focus, depth of focus, and magnification / reduction operations to obtain optimal image quality; and / or 3) utilize AI to identify external engine features to enhance image quality and reconstruct sensors. An inspection system integrating such imaging devices and controls can help standardize external engine inspections into AI-assisted, multi-sensor-based inspections to identify anomalies, construct sensors, detect and identify engine structures, and digitize the entire inspection for customer interaction and future reference. In this way, engine plant visits can be tracked globally during MRO plant visits, and this data can be used to generate analytics that help understand customer usage.

[0101] The inspection system and methods can be used Figures 7A-7H and Figure 8 One or more of the illustrated device configurations are used to ensure consistent or improved sensor positioning. The inspection system may include a gantry or rack with multiple pre-programmed sensors. The inspection system may also include a gantry with multiple pre-programmed sensors that can move via a track. The inspection system may also include a pre-programmed multi-sensor gantry on a track. In another approach, the inspection system may include a robot with one or more pre-programmed sensors on a track. In another approach, the inspection system may include a robot with one or more pre-programmed sensors. In other approaches, the inspection system may include a sky robot with pre-programmed sensors. In other approaches, the inspection system may include a drone for drone-based inspections. In other approaches, the inspection system may include a device for moving the motor when the camera is stationary.

[0102] Now for reference Figure 6This provides a method for automated inspection. Automated inspections can be performed on engines (such as aircraft engines) or their components. This method can be performed based on communication between the inspection computer system 110 and the image capture system 120.

[0103] In step 220, the inspection computer system 110 determines inspection recipes and initial instructions for inspecting the engine and / or its components. In some embodiments, the inspection recipes and initial instructions may be determined manually by an inspection operator. For example, the inspection operator may take images. In some embodiments, the inspection recipes and initial instructions are retrieved from the recipe database 132 based on identifiers associated with the engine and / or its components. For example, the identifier may be a serial number, model number, or classification associated with the engine and / or its components. In some methods, the inspection computer system 110 may obtain the identifier from an image of the engine and / or its components. For example, the identifier may be obtained from an image of a nameplate, or in some aspects, from AI recognition of the nameplate in an image. In some embodiments, the inspection recipes and initial instructions are retrieved from the recipe database based on the engine or component construction, for example, the engine or component construction as determined based on electronic drawing files or models (such as CAD files) associated with the engine and / or its components.

[0104] In other embodiments, the recipe and initial instructions are generated based on a recipe machine learning model trained on multiple image sets captured from multiple parts, physical part models, and / or computer part models. The images in the training set can be labeled with image quality metrics that identify the images and the data quality of the images. (See reference...) Figure 4 Further details of the recipes generated by the machine learning algorithm are described. In some embodiments, the inspection operator can use the user interface device 140 to select the inspection recipe and / or initial instructions for inspection. In this way, machine learning can determine the recipes and inspection instructions (e.g., image capture location and settings) most likely to produce high-quality images.

[0105] In other embodiments, the inspection recipe and initial instructions can be determined via simulation on a 3D model, such as a CAD model. For example, simulation on a 3D model can be used to determine the image capture locations and settings that provide full or adequate coverage of the engine and / or its components.

[0106] The inspection recipe can identify multiple locations for image capture using the image capture system 120 during the inspection work. These multiple locations can be sensor locations for locating one or more sensors of the image capture system 120 within the imaging space. The inspection recipe can define the locations of one or more sensors of the sensor system 124 of the automated image capture device 122 and / or the sensor system 128 of the user-operated image capture device 126. Such sensors can include one or more of optical sensors, LiDAR, 3D scanners, infrared sensors, terahertz spectrometers, microwave imaging sensors, X-ray imagers, computed tomography scanners, eddy current imaging sensors, or ultrasonic sensors.

[0107] The inspection recipe can identify the position of the image capture system 120 relative to an engine, component, or part of a component. For example, the position in the inspection recipe can specify how the automated image capture device 122 or the user-operated image capture device 126 should be positioned relative to the component of interest (e.g., the component to be inspected). In one example, these positions are coordinate positions relative to the inspection space, engine, component, or part of a component. The inspection recipe can also specify the image capture orientation of the sensors of the image capture system 120, such as roll, pitch, and / or yaw. In some embodiments, the inspection computer system 110 can provide the inspection recipe to the automated image capture device 122. In particular, the inspection computer system 110 can provide the inspection recipe (e.g., capture position) to the positioning system 123 of the automated image capture device 122.

[0108] In other embodiments, the inspection device provides an inspection recipe (e.g., capture locations) to a user-operated image capture device 126. The inspection computer system 110 may provide the inspection recipe to the user-operated image capture device 126. The user-operated image capture device 126 may display the inspection recipe to an inspection operator via a user interface 127. In one example, the user interface 127 may list one or more locations for image capture. In another example, the user interface 127 may cover one or more locations for image capture on a 3D model or CAD file of an engine or its components. In this way, the inspection recipe can guide manual image capture using the user-operated image capture device 126.

[0109] Initial instructions may include capture instructions for automated image capture device 122 and / or for user-operated image capture device 126. Capture instructions may be machine instructions for positioning system 123, used to position one or more sensors of the image capture system relative to the engine and / or its components. Capture instructions may also include machine instructions for one or more sensors of sensor system 124. Machine instructions for one or more sensors can provide sensor settings or configurations, such as image capture configurations. For example, when the sensor is a camera, the instructions can provide image capture configurations such as exposure time, focal length, shutter speed, image resolution, light sensitivity setting, f-number, depth of field, focus, zoom level, contrast-to-noise ratio (CNR), signal-to-noise ratio (SNR), illumination settings, aperture, sensor selection, or other settings. Capture instructions may include machine instructions for one or more sensors of sensor system 128. Inspection computer system 110 may transmit machine instructions to image capture system 120. In some embodiments, inspection computer system 110 may transmit instructions to user interface 127, and user interface 127 may display the instructions to an inspection operator.

[0110] In some embodiments, the image capture system 120 may further include an illumination system. Initial instructions may include machine instructions for the illumination system based on a check recipe.

[0111] In step 221, the inspection computer system 110 can identify the position and / or orientation of the engine and / or its components. The position of the engine and / or components can provide the coordinate position of the engine and / or components relative to the inspection space, the engine, or the components. In this way, at step 222, the inspection computer system 110 can identify a capture position based on the position and / or orientation of the engine and / or components. For example, the inspection formula can provide a capture position specifying the position of a sensor (e.g., an image capture device) relative to the component. Therefore, in order to move the sensor into place, the position of the component is also identified so that the sensor can be moved into place. In some embodiments, the position and / or orientation of the engine and / or components is determined via markings on a bracket supporting the engine and / or components. In some embodiments, the position and / or orientation of the engine and / or components is determined based on markings located on or near the component. Markings may include one or more optical markings, color-coded markings, shape-coded markings, pattern-coded markings, embossed markings, engraved markings, sonar-readable markings, and / or lidar-readable markings.

[0112] In step 222, the inspection computer system 110 identifies a capture location. The capture location may include coordinates relative to the inspection space, engine, component, or a portion of the component. For example, the capture location may include the XYZ coordinates of the positioning system 123 or a portion thereof relative to the inspection space, engine, component, or a portion of the component. In some embodiments, the capture location may be provided directly from the inspection recipe. In other embodiments, the capture location may be determined based on the inspection recipe and the component location and orientation.

[0113] In step 223, the inspection computer system 110 determines the movement of the positioning system 123. The inspection computer system 110 can determine the movement of the positioning system 123 to move one or more sensors of the sensor system 124 from an initial position to one or more capture positions. The movement can be a path or a series of coordinate positions of the positioning system 123 or a portion thereof relative to the inspection space, engine, component, or a portion of a component. In one example, the positioning system 123 can move one or more sensors of the sensor system 124 to a capture position. Therefore, the movement can be a movement pattern, path, or series of coordinates of the sensor system 124. The positioning system 124 is moved into place using the positioning system 123. In another example, a positioning system (such as a track system, turntable, etc.) is used to move the engine and / or component into place. Therefore, the movement can also be a movement of the engine or a component thereof. In some embodiments, the inspection computer system 110 determines the capture order of multiple capture positions. Furthermore, the inspection computer system 110 determines machine instructions for the positioning system 123, which can specify the movement of the positioning system 123 according to the capture order.

[0114] In some embodiments, step 223 may be performed by an inspection operator rather than by the inspection computer system 110. For example, a user-operated image capture device 126 may acquire an image at a capture location. The inspection computer system 110 may provide the image capture location to the user-operated image capture device 126, which may display or otherwise communicate the image capture location on a user interface 127. In this way, the inspection operator may determine the movement of the user-operated image capture device 126 and may guide the user-operated image capture device 126 to the capture location.

[0115] In step 230A, the inspection computer system 110 instructs the positioning system 123 of the automated image capture device 122. The inspection computer system 110 may instruct the positioning system 123 to move as determined in step 223 to position one or more sensors of the sensor system 124 in the capture position. In some embodiments, the inspection operator may manually move one or more sensors of the sensor system 128 to the capture position.

[0116] In step 230B, the inspection computer system 110 then instructs one or more sensors of the image capture system 120 to capture images at the capture locations. The inspection computer system 110 may select at least one sensor of the image capture system 120 for each capture location. The inspection computer system 110 may also determine an image capture configuration for each capture location based on the inspection recipe. The inspection computer system 110 may provide image capture configuration instructions to the image capture system 120 based on the image capture configuration.

[0117] In step 240, the inspection computer system 110 can verify one or more images captured by the image capture system 120. In some methods, verification involves determining whether the image meets one or more image requirements. Image requirements may be associated with an inspection recipe. Image requirements may include, for example, general focus, focus of the point of interest, amount of blur, exposure, brightness, overlap with adjacent images, identification of engine and / or component parts in the overlapping portion, presence of engine and / or component parts, etc. Image requirements may include one or more image quality metrics and may include any relevant image quality metrics.

[0118] In some embodiments, the inspection computer system 110 may compare one or more features of an image captured by the image capture system 120 with inspection requirements. For example, the inspection computer system 110 may compare the brightness level of the image with the desired brightness level specified in the inspection requirements.

[0119] In some embodiments, the inspection computer system 110 may use machine learning to verify images. The machine learning model 136 may include one or more models trained using historical images labeled with inspection requirements (e.g., points of interest, blur amount, exposure, brightness, etc.). For example, the inspection verification training dataset may include labeled captured images to indicate whether the captured images meet one or more inspection requirements. The inspection verification training dataset may include multiple captured images, each labeled with an inspection location (e.g., indicating the capture location used to acquire the captured image), a capture configuration (e.g., indicating the sensor configuration used to acquire the captured image), and one or more inspection requirement indicators (e.g., indicating whether the captured image meets one or more inspection requirements). In some embodiments, the inspection verification training dataset may be stored as training data 138. The inspection computer system 110 may use the inspection verification training dataset to establish a correlation between features of images that meet inspection requirements (e.g., satisfactory) and features of images that do not meet inspection requirements (e.g., defective). In this way, the trained machine learning model may automatically identify whether an image meets inspection requirements based on this correlation. The trained machine learning model may receive images of the engine and / or its components captured at one or more capture locations as input. The trained machine learning model can identify whether an image is satisfactory or defective relative to one or more inspection requirements, as output.

[0120] In step 245, the inspection computer system 110 determines updated instructions for inspection. The inspection computer system 110 may determine the updated instructions based at least in part on the image verification results. The updated instructions may identify one or more revised or additional capture locations of the image capture system 120. Additionally, the updated instructions may include additional and / or revised capture configurations or settings of the sensors of the image capture system 120. Furthermore, the updated instructions may instruct the use of one or more additional sensors (e.g., sensors of different types or capabilities) for additional image capture. The inspection computer system 110 may determine the revised capture configuration based on image quality metrics. In a non-limiting example, if the verification results indicate a defective CNR in one or more images, the updated instructions may adjust the depth of focus as the F-stop increases. In another example, if the verification results indicate the presence of a part of an engine and / or component in the image, or a part of a component (e.g., a nameplate), the updated instructions may instruct the camera to reposition itself in one or more quadrants of the image including the part, and may also instruct the camera to zoom in or adjust focus to obtain a more detailed image of the part.

[0121] Updated instructions are provided to image capture system 120. In this way, the updated instructions may instruct positioning system 123 to move to an additional or revised capture position. Inspection computer system 110 may provide machine instructions to positioning system 123 based on the revised capture position. Furthermore, the updated instructions may also instruct one or more sensors of sensor system 124 and / or sensor system 128 to capture images using a revised capture configuration or settings. Inspection computer system 110 may also provide machine instructions to sensor system 124 and / or sensor system 128 based on the revised capture configuration. In some embodiments, inspection computer system 110 may instruct image capture system 120 to adjust the capture configuration to recapture the image.

[0122] In some embodiments, the inspection computer system 110 may retrieve updated instructions from the inspection database 130 based on defect inspection requirements. For example, the inspection database 130 may include updated instructions associated with defect inspection requirements. The updated instructions may be instructions aimed at obtaining satisfactory images and meeting inspection requirements previously identified as defects associated with the engine and / or engine components.

[0123] In some embodiments, the inspection computer system 110 may use machine learning to determine updated instructions for inspection. The machine learning model 136 may include one or more models trained using historical images labeled with inspection requirements (e.g., points of interest, blur amount, exposure, brightness, etc.) and / or labeled with updated instructions that satisfy the defect inspection requirements. For example, the instruction training dataset may include captured images labeled with one or more defect inspection requirements and with instructions to be updated to satisfy those requirements. In some aspects, the instruction training dataset may include multiple captured images, each labeled with an inspection location, capture configuration, and one or more updated instructions. The instruction training dataset may be stored as training data 138. The inspection computer system 110 may use the instruction validation training dataset to establish a correlation between defect inspection requirements (and / or image features) and updated instructions. In this way, the trained machine learning model may automatically identify updated instructions for the image capture system 120 based on this correlation. The trained machine learning model may receive images of the engine and / or its components and / or image-related defect inspection requirements as input. The trained machine learning model may identify updated instructions for inspection as output.

[0124] In some embodiments, the adjusted capture configuration is determined based on a machine learning model trained with an image quality training dataset. The machine learning model may be one of machine learning models 136, and the image quality training dataset may be stored as training data 138. The image quality training dataset includes multiple captured images, each labeled with a capture location, capture configuration, and quality metric.

[0125] Figures 7A to 7H An exemplary positioning system is shown that can be used as a positioning system 123 in an image capture system 120. Figures 7A-7H One or more of the positioning systems shown can be used to position one or more sensors relative to component 105 (such as an engine or a component thereof).

[0126] Figure 7A A gantry 123A or rack is shown. The gantry 123A spans component 105. One or more sensors of the image capture system 120 are mounted to the gantry 123A. In some embodiments, the gantry 123A may include a crane or other means capable of lifting and / or repositioning component 105. The gantry 123A may be configured to move one or more sensors of the image capture system 120 around or relative to component 105 or its components.

[0127] Figure 7B A gantry 123B-1 on track system 123B-2 is shown. Gantry 123B-1 spans component 105. One or more sensors of image capture system 120 are mounted to gantry 123B-1. In some embodiments, gantry 123A may include a crane or other means capable of lifting and / or repositioning component 105. Gantry 123B-1 is configured to move one or more sensors of image capture system 120 around or relative to component 105 or its components. Gantry 123B-1 may move along track 123B-2 to reposition the gantry and one or more sensors on it relative to component 105. Gantry 123B-1 is configured to move one or more sensors of image capture system 120 in a first plane. Track system 123B-2 is configured to move gantry 123B-1 in a direction perpendicular to the first plane.

[0128] Figure 7C A powered vehicle 123C with sensor 124C is shown. Sensor 124C may include one or more sensors of image capture system 120. Powered vehicle 123C is movable. Moving powered vehicle 123C can reposition sensor 124C around component 105 or its components. Powered vehicle 123C may include a programmable robotic arm configured to position sensor 124C.

[0129] Figure 7D A cable-suspended camera positioning system is shown, comprising a camera system 124D suspended by a cable 123D. The cable 123D is configured to move and reposition the camera system 124D. Operating the cable 123D allows the camera system 124D to move relative to a component 105 located below the cable 123D.

[0130] Figure 7E A component 105 is shown positioned within an imaging gantry 123E. The imaging gantry 123E includes multiple gantries. Each gantry is coupled to one or more sensors. Each gantry 123A can be configured to move one or more sensors of the image capture system 120 around or relative to component 105 or its components.

[0131] Figure 7F A powered vehicle 123F-1 on the orbital system 123F-2 is shown. The powered vehicle 123F-1 includes a sensor 124F. The sensor 124F may include one or more sensors from the image capture system 120. The powered vehicle 123F-1 may include a programmable robotic arm configured to position the sensor 124F.

[0132] Figure 7G A drone 123G including sensor 124G is shown. Sensor 124G may include one or more sensors of image capture system 120. Drone 123G may be configured to autonomously orient sensor 124G relative to component 105.

[0133] Figure 7H A component 105 is shown positioned on a turntable 124H-2. Component 105 is mounted on a support 124H-1 that supports component 105. A sensor 124F may be mounted adjacent to the turntable 124H-2. The turntable 124H-2 is configured to allow component 105 to rotate on an axis. Sensor 124F may include one or more sensors of the image capture system 120. In some embodiments, sensor 124F is mounted to a programmable robotic arm configured to position sensor 124F.

[0134] Figure 8 An exemplary embodiment of an image capture system 705 is shown. The image capture system 705 can be used as an image capture system 120. The image capture system 705 includes a frame 720 supporting a member 105 and a rotating tool 745. The member 105 is suspended from the frame 720 via a cable. In some embodiments, the rotating tool 745 is configured to rotate a part of the member, such as a turbine of an engine. In some embodiments, the rotating tool 745 is configured to rotate or rotate the member 105 on the frame 720.

[0135] The image capture system 705 also includes an endoscopic inspection (BSI) device with a robotic arm 741 for positioning one or more sensors 742. One or more sensors 742 are coupled to the distal end of the robotic arm 741. The robotic arm 741 is coupled to a base 740. The base 740 may be a cabinet housing one or more accessories for the robotic arm 741 and / or one or more sensors 742, and a controller 744. For example, the base 740 may store batteries, routers, sidecars, etc. A user interface 743 may also be coupled to the base 740.

[0136] The image capture system 705 also includes an engine inspection service device. The EIS device includes a robotic arm 731, to which one or more sensors 732 are coupled. The EIS device also includes a base 730. The robotic arm 731 and a user interface 733 are coupled to the base 730.

[0137] The image capture system 705 also includes an autonomous mobile robot (AMR) 710. The AMR 710 is configured to move independently (e.g., without a track or operator supervision) through the inspection space around component 105. The AMR 710 may also include one or more sensors coupled to it.

[0138] The inspection system and methods described herein also allow for dynamic adaptation to the initial operating range of engine MRO. The inspection methods can be adapted to the initial operating range of engine MRO. Furthermore, the inspection methods can guide imaging techniques for different inspection situations and / or scenarios that may arise in the field.

[0139] In some methods, dynamic recipe generation methods can further and / or inquire into the identified regions of interest (ROIs).

[0140] In one scenario, condition assessment AI can identify specific predefined ROIs for further review, such as missing parts, potential signs of defects, or incorrectly positioned parts. This method can automatically generate robot instructions to move an inspection robot (such as a robotic arm equipped with cameras or other sensors, e.g., depth sensing) to the ROI and collect additional data. This additional data can be supplementary images or information (e.g., depth information) from sensors at different angles around the ROI under potentially varying lighting conditions, to obtain additional details about the ROI.

[0141] The generation of instructions for acquiring additional data can be automated and / or AI-driven. Automation of instruction generation can be achieved, for example, through offline learning via deep reinforcement learning. In some approaches, the inspection robot can learn how to move from “bad” or low-quality image frames to “good” or high-quality image frames. In another approach, the inspection robot can learn how to adjust lighting and / or camera parameters to improve image quality.

[0142] In another scenario, the inspection method can include particularly detailed inspections. For example, human input can initiate a particularly detailed inspection by asking a human inspector for random ROIs deemed necessary. This method can iteratively capture image and / or other sensor data about the ROI, aiming to obtain high-quality data for multiple points, and in some respects, high-quality data for each individual point within the ROI. Quality metrics can be predefined and can include pixel-level metrics such as sharpness, blurriness, etc. Dynamic recipe generation methods can dynamically and iteratively generate recipes to ensure that all ROIs contain data that meets the quality metrics.

[0143] In other approaches, automated recipe generation methods can guide consistent imaging based on various conditions. For example, a condition may assign or automatically determine inspection limitations or standards for a specific part number and / or a given part number, and automatically generate a recipe based on this information. This method can programmatically retrieve information related to a given part number, including but not limited to: a) manuals of recommended work areas and / or recipes for several potential indications at different areas of the part; b) historical inspection records of the given part number at locations and / or areas that have already been inspected using the corresponding imaging recipe; and c) conditional data patterns and / or heat maps of areas with frequent, detailed inquiries from an airline, service plant, flight routes, etc. This method can combine these different types of retrieved information and automatically generate a series of imaging recipes based on the retrieved information.

[0144] The inspection methods described herein can be applied to internal (e.g., endoscopic) inspections, fuselage inspections, factory inspections, external inspections, etc. These methods can be applied to all engine types and all components with maintenance and repair requirements involving inspection-based work scope planning. It is envisioned that the inspection methods described herein can improve MRO and / or factory utilization by reducing the time required to obtain customer authorization, and can reduce or prevent factory workflow disruptions. The inspection methods can also provide customers with visibility into repair workflows. Furthermore, the inspection methods can establish customer-specific parameters for work scope planning.

[0145] Figure 9An exemplary method for dynamic recipe generation to provide detailed queries of predefined regions of interest is illustrated. This method can be performed on an engine (such as an aircraft engine) or a component thereof. The method can be executed based on communication between an inspection computer system 110 and an image capture system 120.

[0146] At step 220, the inspection computer system 110 determines an initial working range for inspection. The initial working range is based on an inspection recipe. The inspection recipe may identify a component or part of a component as the target of inspection. The inspection recipe may include multiple capture locations that specify the locations where the image capture system 120 will capture images of the component or part. The inspection recipe may also include image data requirements for the images captured as part of the initial working range. The initial working range may also include initial instructions, which may include capture instructions for the image capture system 120. Capture instructions may be machine instructions of the positioning system 123 for positioning one or more sensors of the image capture system relative to the engine and / or its components. Capture instructions may also include machine instructions of one or more sensors of the sensor system 124, for example, providing capture configuration or settings.

[0147] In some embodiments, the inspection recipe may be determined manually by the inspection operator. For example, the inspection operator may take images. In some embodiments, the inspection recipe is retrieved from the recipe database 132 based on an identifier associated with the engine and / or engine components. For example, the identifier may be a serial number, model number, or classification associated with the engine and / or its components. In some methods, the inspection computer system 110 may obtain the identifier from an image of the engine and / or its components. For example, the identifier may be obtained from an image of a nameplate, or in some respects, from AI recognition of the nameplate in an image. In some embodiments, the inspection recipe and initial instructions are retrieved from the recipe database based on the engine or component construction, such as an engine or component construction determined based on electronic drawing files or 3D models (such as CAD files) associated with the engine and / or its components. The inspection recipe may also be determined based on engine inspection history, engine construction records, engine repair history, customer-specified task requirements, and / or identified problems.

[0148] In other embodiments, the inspection recipe is generated based on a recipe machine learning model trained on multiple images captured from multiple engines, multiple components, physical component models, and / or computer component models. Images in the training set can be labeled with image quality metrics that identify the images and the data quality of the images. The multiple images in the training set may also be labeled with component identifiers, component repair history, geographic regions associated with the component, component flight path history, and / or component operator identifiers. The multiple images in the training dataset may include images of components of the same or similar types, images of component models, and / or images from computer simulations of the components. Reference Figure 4 Further details of the recipes generated by the machine learning algorithm are described. In some embodiments, the inspection operator can use the user interface device 140 to select the inspection recipe and / or initial instructions for inspection. In this way, machine learning can determine the recipes and inspection instructions (e.g., image capture location and settings) most likely to produce high-quality images.

[0149] In other embodiments, the inspection recipe and initial instructions can be determined via simulation on a 3D model, such as a CAD model. For example, simulation on a 3D model can be used to determine the image capture locations and settings that provide full or adequate coverage of the engine and / or its components.

[0150] In other embodiments, the check recipe is generated based on a mapping with a computer vision algorithm, wherein reference images in the maintenance manual are associated with computer models of the components.

[0151] In step 230, the inspection computer system 110 communicates instructions to the image capture system 120 based on an initial working range. In some embodiments, the inspection computer system 110 communicates instructions to the positioning system 123 of the automated image capture apparatus 122. The inspection computer system 110 may instruct the positioning system 123 to move one or more sensors of the sensor system 124 to a capture position identified as part of the working range in the inspection recipe. In some embodiments, the inspection operator may manually move one or more sensors of the sensor system 128 to the capture position. In some embodiments, the inspection computer system 110 instructs one or more sensors of the image capture system 120 to capture an image at the capture position. The inspection computer system 110 may select at least one sensor of the image capture system 120 for each capture position. The inspection computer system 110 may also determine an image capture configuration for each capture position based on the inspection recipe. The inspection computer system 110 may provide image capture configuration instructions to the image capture system 120 based on the image capture configuration.

[0152] In step 235, the computer system 110 is checked to receive captured images. The computer system 110 may receive one or more captured images taken as part of an initial working range.

[0153] In step 247, the inspection computer system 110 identifies trigger conditions based on the detection or recognition of captured images. Trigger conditions can be any conditions that ensure one or more adaptation tasks are performed as part of the inspection scope. Adaptation tasks may include capturing images at one or more additional capture locations using an adjusted capture configuration or setup, and / or using a new sensor (e.g., a different type of sensor). For example, adaptation tasks may be ensured to obtain a complete set of inspection data, acquire images of acceptable quality, or perform detailed integration of regions of interest (such as areas where anomalies are detected). In some embodiments, trigger conditions are identified by comparing captured images to image data requirements specified in the inspection recipe.

[0154] In some embodiments, the triggering condition includes captured images that fail verification. The checking computer system 110 may be configured to verify the image quality of captured images and identify one or more images that fail verification. Verification may involve checking one or more image quality metrics, such as image resolution, illumination, sharpness, and frame rate of the region of interest.

[0155] In some embodiments, machine learning can be used to detect trigger conditions. Trigger conditions can be detected based on conditional evaluation of the part using a machine learning model trained on sample inspection images, and the conditions are associated with the sample inspection images. Trigger conditions can also be detected using a trigger condition machine learning model trained on multiple images labeled with trigger conditions. The trigger condition training dataset can be stored as training data 138 and includes multiple images labeled with trigger conditions. At step 910, the inspection computer system 110 processes the trigger condition training data. At step 911, the inspection computer system 110 trains a machine model using the trigger condition training data to develop a trigger condition model 136C. The trigger condition model 136C can receive one or more of the captured images as input. Furthermore, the trigger condition model 136C can automatically identify one or more trigger conditions as output.

[0156] In other embodiments, triggering conditions may be detected based on component identifiers, component repair history, geographic regions associated with component use, component flight path history, and / or component operator identifiers. For example, a component repair history indicating that a part of the component has been previously repaired or replaced may be a triggering condition that warrants performing one or more adaptation tasks to capture additional images of the part.

[0157] In some embodiments, the triggering condition includes an anomaly detected in a previously captured image. For example, the region of interest may correspond to a region based on comparing the captured image with a reference image and / or based on a reference. Figure 5 The described condition assessment evaluates anomalies detected in the image. One or more adaptation tasks can be performed to capture additional images of the anomalous regions to better assess the condition of the part.

[0158] In step 257, the inspection computer system 110 determines an adaptation task for inspection and instructs the image capture system 120 to perform the adaptation task. The adaptation task may include an image capture task at a new capture location. In some methods, the new capture location is determined based on an estimated location of a region of interest determined based on one or more captured images. In some methods, the new capture location may be determined based on simulating the movement of the positioning system 123 of the image capture system 120 using a computer model of the components. The adaptation task may also include an image capture task utilizing a new capture configuration and / or utilizing a new sensor (e.g., a different type or sensor with different capabilities). The adaptation task may also include a detailed interrogation task for the identified region of interest. In some methods, the adaptation task may also include a task of capturing multiple images with different capture configurations. The adaptation task may also include a task of capturing multiple images at multiple different capture locations and / or angles. Furthermore, the adaptation task may include a set of predefined capture locations around the identified region of interest.

[0159] The adaptation task can be a task that provides updated instructions to the image capture system 120. In one example, the captured image is captured by a first sensor of the image capture system 120, and the updated instructions are configured to cause a second sensor of the image capture system 120 to capture the image. The new capture configuration may include one or more of a new zoom level, exposure time, light sensitivity setting, illumination setting, image resolution, video length, and sensor type selection. In another example, the captured image is captured using the first capture configuration, and the updated instructions are configured to cause the image capture system 120 to capture the image using the second capture configuration. The examining computer system 110 can determine the updated instructions based on a positioning system motion machine learning model. The positioning system motion machine learning model can be configured to determine a path from a first capture location of the captured image to a second capture location for the adaptation task. The first capture location may be the location where the captured image is acquired. The second capture location is a new location that the positioning system can move to according to the adaptation task. In some methods, the positioning system motion machine learning model can be trained using captured images labeled with the first capture location, the second capture location, and a movement or motion path that moves the sensor from the first capture location to the second capture location.

[0160] In steps 270 and 278, the inspection computer system 110 identifies maintenance, repair, and overhaul (MRO) tasks and provides instructions to perform the MRO tasks. The inspection computer system 110 may provide instructions to the MRO system 141 to perform the MRO tasks. An MRO task may be a repair or maintenance task on a component or a part thereof.

[0161] In some embodiments, the inspection computer system 110 may be configured to provide an inspection user interface to a user on one or more of the user interface devices 140. The inspection computer system 110 may transmit captured images and / or adaptation tasks to the reviewer user interface for display. Additionally, the inspection computer system 110 may determine instructions for the adaptation tasks based at least in part on user input received via the reviewer user interface.

[0162] In step 280, the inspection computer system 110 may receive feedback regarding the triggering conditions, adaptation tasks, and / or MRO tasks of the captured images. The inspection computer system 110 can then use this feedback to update the training dataset and store it as further training data 138. For example, this feedback loop can provide real-time input to the training dataset to fine-tune machine learning models, such as trigger-conditional machine learning models.

[0163] Go to Figure 10 This illustrates a method for generating dynamic recipes that provide detailed queries for a predefined region of interest. This method can be performed on an engine (such as an aircraft engine) or its components. The method can be executed based on communication between an inspection computer system 110 and an image capture system 120.

[0164] At step 1010, the inspection computer system 110 initiates an initial inspection of the engine MRO. The initial inspection can be for the exterior or interior of the engine, and in some respects, can focus on one or more components of the engine. At box 1020, the inspection computer system 110 can use one or more items to determine a recipe for the initial engine inspection. As shown in box 1020, one or more of the following can be used to determine or generate a recipe for the initial inspection: factory manual, recommended work scope, recommended recipe, historical inspection records (e.g., by part number, serial number, part category, etc.), and conditional data patterns or heatmaps (e.g., by airline, factory, route, etc.).

[0165] Steps 1030-1033 detail a method for identifying regions of interest using a machine learning model for detailed inquiry. The method in steps 1030-1033 utilizes machine learning based on captured images obtained from an initial inspection recipe.

[0166] At step 1030, the inspection computer system 110 identifies one or more predefined regions based on the recipe. The recipe provides a set of predefined regions of interest for identifying portions of the engine for detailed inquiry. The inspection computer system 110 can then automatically identify one or more additional regions of interest from the predefined regions of the engine for detailed inquiry using the methods described below.

[0167] At step 1031, the computer system 110 is checked to cause the image capture system 120 to execute a set of predefined recipes for each region of interest. In this way, the image capture system 120 captures multiple images and / or data in a detailed interrogation of each region of interest.

[0168] At step 1032, the computer system 110 uses a data quality inspector to analyze the captured image and / or data to determine whether the captured image provides sufficient or complete coverage of the region of interest. In some embodiments, the data quality inspector refers to... Figure 6 The method described. If the data quality is insufficient, the method may return to step 1031 to perform additional image and / or data capture of the region of interest. If the data quality is sufficient, the method performs a conditional evaluation at step 1033.

[0169] At step 1033, the inspection computer system 110 may also use conditional evaluation AI to analyze the captured images. The conditional evaluation AI may utilize an ROI machine learning model to identify further regions of interest to be queried based on the captured images. In some methods, the training dataset includes multiple captured images labeled with additional regions of interest, and in some methods, recipes for the queries associated with the additional regions of interest. In this way, the ROI machine learning algorithm can receive captured images as input and identify further regions of interest and recipes for examining the further regions of interest as output.

[0170] Steps 1043-1052 describe in detail a method that uses input from an inspection operator to drive the identification of a particular area for detailed inquiry.

[0171] At step 1045, the computer system 110 is examined to identify one or more regions of interest (ROIs) based on a recipe. The recipe provides a set of predefined regions for identifying parts of the engine for detailed inquiry.

[0172] At step 1040, the inspection operator and / or inspection computer system 110 may identify or generate a recipe for each region of interest to obtain captured images and / or data during detailed inquiry. In some embodiments, the inspection operator may use the image capture system 120 for positioning and pose estimation to capture multiple images at various capture locations and utilizing various capture configurations of the sensor. In other embodiments, the inspection computer system 110 may automatically instruct the image capture system 120 to capture multiple images, wherein the capture locations and capture configurations are predefined for a specific region of interest.

[0173] At step 1052, the computer system 110 uses a data quality inspector to analyze the captured image and / or data to determine whether the captured image provides sufficient or complete coverage of the region of interest. In some embodiments, the data quality inspector refers to... Figure 6 The method described. If the data quality is insufficient, the method may return to step 1040 to capture additional images and / or data of the region of interest. Therefore, the recipe at step 1040 can be iterative and may involve capturing additional images and / or data until a data quality check determines that enough images and / or data of the region of interest have been collected. When the data quality is sufficient, the method proceeds to step 1043.

[0174] At step 1043, one or more human operators may evaluate the captured images and / or data to determine whether further detailed inquiry should be conducted in one or more additional regions of interest. In some embodiments, an examiner user interface may be displayed on one or more user interface devices 140. The examiner user interface may display captured images and / or data for each acquired region of interest. In some methods, a 3D model or engineering drawing that can map the captured images to the engine or its components is displayed. The examiner operator may then input instructions into the examiner user interface to inquire about additional regions of interest. In this way, the examiner computer system 110 may determine instructions for further inquiry into regions of interest and communicate such instructions to the image capture system 120.

[0175] Figures 11A-11C Examples of images acquired using the image capture system 120 are shown. These images show captured images from the initial inspection recipe, as well as additional images acquired for detailed inquiry to identify regions of interest and / or acquired using modified capture configurations to provide improved data quality.

[0176] Figure 11ASensor 1115A-1, which captures an image of component 105, is shown. Sensor 1115A-1 is a camera with a wide field of view. Distance sensor 1115A-2 is associated with sensor 1115A-1. Distance sensor 1115A-2 determines the distance from sensor 1115A-1 to component 105. This distance determines the initial capture configuration of sensor 1115A-1. In some methods, input from a 3D model or engineering drawing of the engine can be used to determine the initial capture configuration of sensor 1115A-1. Sensor 1115A-1 uses the initial capture configuration to capture a first image 1105 of component 105. Here, the initial capture configuration includes a large depth of field, a low aperture, and initial focus settings. A machine learning model (e.g., a trigger-condition machine learning model and / or an ROI machine learning model) analyzes the first image 1105 and identifies an adaptive task including instructions for capturing a second image 1110. Sensor 1115A-1 receives machine instructions for the adaptation task and captures a second image 1110 of component 105. The second image includes an updated capture configuration and an updated capture position. The updated capture configuration adjusts the zoom level (e.g., magnification on the region of interest) and focus settings of sensor 1115A-1. The adaptation task provides instructions for detailed interrogation of the region of interest 1120 identified using the ROI machine learning model.

[0177] Figure 11B A first sensor 1115B-1 and a second sensor 1115B-2 are shown for capturing images from the capture unit 105. The first sensor 1115B-1 is a camera with a wide field of view. The second sensor 1115B-2 is a camera with a narrow field of view. The first sensor 1115B-1 captures a first image 1125 of the capture unit 105 using an initial capture configuration. Here, the initial capture configuration includes a large depth of field, a low aperture, and an initial focus setting. A machine learning model (e.g., a trigger-condition machine learning model and / or an ROI machine learning model) analyzes the first image 1125 and identifies an adaptive task that includes instructions to capture a second image 1130 using the second sensor 1115B-2. The instructions include a movement to reposition the second sensor 1115B-2 to capture an image focused on the region of interest 1120, which includes a nameplate.

[0178] Figure 11C Sensor 1115C is shown, capturing an image of component 105. Sensor 1115C captures a first image 1135 of component 105 with an initial configuration. The computer system 110 is examined using a reference. Figure 6The described method evaluates image quality. A first image 1135 has a first region 1137A and a second region 1137B, where the first region 1137A has a low CNR and the second region 1137B has a high CNR. A computer system 110 identifies an update capture command that adjusts the depth of field as the F-stop increases and communicates the command to a sensor 1115C to capture a second image 1140. The second image 1140 has a high F-stop to increase the depth of field.

[0179] The inspection systems and methods described herein may also include an Engine Inspection Review and Control Center (RaCC). RaCC facilitates digital control and review of multi-modal inspection devices and data types, automatically associating with electronic manuals and AI-assisted layout determination for engine, module, and piece-part level inspections. RaCC may also include large-format image review to leverage the inspector's ergonomic and visual advantages. RaCC may also include automated report generation, saving hours of manual formatting and organizing of relevant layout information. In some methods, RaCC can also enable fully digital inspection data logging that provides historical context for defect patterns.

[0180] RaCC enables command and control of a complete set of engine, module, and component inspection devices as part of an integrated, standardized, and automated inspection system.

[0181] Inspection systems and methods may include device handling and / or menu-guided control via RaCC for engine inspection. Using RaCC, the inspection system can move a mechanized engine rack into place using a mobile robot and / or a gantry-based system, and / or retrieve and position engine imaging equipment. It can also be a central hub for guiding endoscopic inspections (BSI) of the engine's turbocharger, compressor, and turbine. For example, once an inspection sequence begins, RaCC can rotate the engine core via a wireless accessory gearbox (AGB) motor.

[0182] RaCC can include human-assisted tools such as augmented reality, computer vision for image acquisition and localization, and remote assistance.

[0183] Following the processing of data acquisition, RaCC can also facilitate the automated collection and presentation of data within the review center. Raw inspection data can be processed into useful information such as attribute tags, defect handling, serial number records, and ERP integration.

[0184] Furthermore, RaCC can integrate electronic engine and / or aircraft manuals with inspection sequences without requiring separate workstations. The applicability of RaCC may extend beyond repair or overhaul shops, reaching aircraft maintenance facilities and wings.

[0185] Go to Figure 12 The diagram illustrates a method for inspecting controls. This method can be used to inspect engines (such as aircraft engines) or their components. The method can be performed based on communication between the inspection computer system 110, the image capture system 120, and the user interface device 140.

[0186] In step 235, the computer system 110 receives the captured image and metadata attached to the captured image. The captured image may be of a component of the aircraft or a part of a component. The computer system 110 receives the captured image and attached metadata from the image capture system 120. The metadata may identify a view of a component or part. In some aspects, the metadata may also identify the location of the captured image or a part identifier.

[0187] In step 1212, the inspection computer system 110 receives supplementary information. In some methods, the inspection computer system 110 is communicatively connected to and receives supplementary information from the supplementary information database 1210. The supplementary information can be any data related to the component and / or inspection process. In some aspects, the supplementary information may include 3D models, manuals, technical specifications, repair options, maintenance instructions, replacement instructions, historical captured images, historical inspection data, or similar information associated with the component or parts of the component.

[0188] In some embodiments, the metadata identifies a view or part of a component, and supplementary information includes technical data and / or maintenance manual data associated with the part of the component. In some embodiments, the metadata identifies a problem with a part of the component, and supplementary information includes repair options associated with the problem.

[0189] In step 1213, the inspection computer system 110 displays controls in a user interface via one or more of the user interface devices 140. These controls may be used for one or more components of the image capture system 120. In some methods, the controls of the image capture system 120 are displayed as an overlay on a view of the sensor system 124 and / or the sensor system 128. The inspection computer system 110 may also display captured images acquired by the image capture system 120, additional metadata, and supplementary information. In particular, the inspection computer system 110 provides an inspection user interface on the user interface device 140. The user interface device 140 may be any suitable interface device, and in some embodiments, may be a virtual, augmented, or mixed reality display. The user interface device 140 may be a head-mounted display (see, for example...) Figure 16 Alternatively, it could be a display screen with an associated keyboard, numeric keypad, touch panel, control column, and / or joystick (see example...). Figure 17In some methods, the user interface device 140 includes a control column or lever for controlling the movement of the positioning system 123 or for reviewing images captured by the image capture system 120. The inspection user interface includes controls for multiple sensors of the image capture system 120 and multiple positioning devices in the positioning system 123 of the image capture system 120. In some embodiments, the controls may be selectively displayed simultaneously on a display of the inspection user interface. For example, the controls may be selectively displayed based on the inspection user interface, based on the parts of the component being inspected, based on the inspection task being performed, and / or based on the inspection recipe or inspection instructions. In some examples, the controls may be selectively displayed based on real-time images captured by the image capture system 120.

[0190] In some embodiments, the inspection user interface includes a representation of aircraft components and a representation of sensors of the image capture system 120. The sensors of the image capture system 120 can be dragged and placed around the representation of components. For example, the inspection operator can drag the representation of sensors to place the sensors at various target locations on the representation of components.

[0191] In some embodiments, the positioning system 123 includes an engine rotation device configured to rotate a part within an engine or the engine itself. The part may include turbine blades, a disk, a bladed disk, and / or a shaft. The user interface may include a first slider for controlling the rotation of the engine rotation device. In some embodiments, the positioning system 123 includes a component positioning system and a sensor positioning system.

[0192] In some embodiments, the user interface includes an image of an engine or a component of an engine for selecting the insertion location of the endoscope.

[0193] In other embodiments, the inspection computer system 110 may retrieve a captured image with additional metadata from the image capture system 120 and selectively display a subset of available controls for the sensors of the positioning system 123 and sensor system 124 based on the metadata of the captured image. The subset of available controls may be based on the location indicated in the captured image and / or part identifiers in the metadata.

[0194] At step 1214, the inspection computer system 110 receives user input for the inspection task. The user input can be received from a user (such as an inspection operator) via an inspection user interface. The inspection computer system 110 can be configured to simultaneously control the movement of the component positioning system and the sensor positioning system to influence the relative position of the sensor system and the component, thereby enabling image capture. Therefore, the user input can be any input that implements the component positioning system and / or the sensor positioning system. The user input can be motion input captured by a motion sensor.

[0195] In some embodiments, when the user input device is a head-mounted display, the user input may include the movement and orientation of the head-mounted display. The inspection computer system 110 may be configured to control the movement of the image capture system 120 (e.g., a component positioning system or sensor positioning system) based on the movement and orientation of the head-mounted display.

[0196] In some embodiments, user input may include touch input received on a touch-sensitive display showing an image of a component or part of a component. The inspection computer system 110 may be configured to determine one or more capture locations based on the touch input, and to determine movement of one or more sensors of the image capture system 120 based on the one or more capture locations. Touch input may include tapping actions, dragging actions, and / or multi-touch squeezing or stretching actions. In some methods, a dragging action may correspond to movement in a plane parallel to the image of the component, and a squeezing or stretching action may correspond to forward or backward movement relative to that plane.

[0197] In some embodiments, the inspection user interface may display previously captured images or models of the component. User input may include user selection of a region of interest in the previously captured image. The inspection computer system 110 may then determine a capture location and / or capture configuration based on the region of interest. The capture location and capture configuration may be selected to focus and / or zoom in on the region of interest.

[0198] At step 1215, the inspection computer system 110 determines and communicates machine instructions. The inspection computer system 110 can determine and communicate machine instructions to the image capture system 120 or any part thereof. Machine instructions may be machine instructions for the positioning system 123 to move the sensor system 124 to one or more capture positions. Machine instructions may also be configuration or setup instructions for the sensor system 124 or sensor system 128 of the image capture system 120. The inspection computer system 110 may also determine and communicate instructions for the component positioning system or sensor positioning system. In this way, the machine instructions enable image capture and generate additional images and additional metadata for step 235.

[0199] Figure 13 A method for generating an inspection report is shown. This method can be performed based on communication between the inspection computer system 110 and the user interface device 140.

[0200] In step 1310, the inspection computer system 110 determines the user role. The user role can be an inspector, administrator, customer, vendor, or auditor. In some embodiments, the user can select on a user interface associated with the user interface device 140 to determine the user role. In other embodiments, the user role can be determined based on the inspection task or based on captured images, inspection tasks, and / or MRO tasks determined during the inspection.

[0201] In step 1320, the computer system 110 retrieves a report template from the template database 1350. The report template may include information related to a specific user role. In this way, the system described herein can customize reports and the information presented therein according to a specific user role. For example, the report template may specify the types of metadata to be included in the report. In another example, the report template may specify the selection of images / part data to be included in the report based on the evaluation criteria of the part. In yet another example, the report may specify whether an image should be included in the report.

[0202] In some embodiments, at step 1355, a report template is generated by a report template machine learning model. The report template machine learning model may be one of machine learning models 136. The report template machine learning model is trained on historical reports for multiple roles. The report training dataset may be stored as training data 138 and may include historical reports tagged with roles and one or more information items known to be associated with those roles. The report training dataset is used to train the report template machine learning model. The report template machine learning model is used to generate report templates, which may be stored in a template database 1350.

[0203] In step 1330, the inspection computer system 110 populates the report template with data and / or supplementary information from the inspection. The inspection computer system 110 may optionally populate the report template with captured images and additional metadata stored in the inspection database 130. The inspection computer system 110 may retrieve inspection information from the inspection database 130, such as captured images, metadata, or other data acquired during the inspection. The inspection computer system 110 may retrieve supplementary information from the supplementary information database 1210.

[0204] In step 1340, the inspection computer system 110 generates a report. This report may be provided to users having one or more roles as described herein. In some embodiments, the inspection computer system 110 may display the report on the user interface of the user interface device 140.

[0205] At step 1345, the computer system 110 is checked to ensure it can receive feedback regarding the report template. This feedback may be user feedback indicating which information is relevant to a specific role. The feedback can be incorporated into the report training dataset as training data 138 and can be used to further refine the report template machine learning model. Users can provide feedback, for example, via the user interface of the user interface device 140.

[0206] Figure 14 A schematic diagram is shown illustrating information provided via a user interface 1410 according to some embodiments. In some embodiments, Figure 14 The user interface can be implemented in processor-based user interface devices (such as...) Figure 1 The user interface device 1410, which communicates with the inspection computer system 110, is provided. The inspection user interface 1410 may include options for internal inspection 1415 or external inspection 1420 of a component. Internal inspection 1415 or external inspection 1420 may include multiple options for representing inspection data on the inspection user interface 1410. The inspection user interface 1410 may include an image overlay 1425 on a model (such as a CAD model) with inspection-based inspection data or other annotations regarding condition evaluation. The inspection user interface 1410 may also include a virtual or augmented reality environment view 1430 of the engine condition or configuration. Users can manipulate the virtual or augmented reality representation of the engine to focus on specific areas of interest. The inspection user interface 1410 may also include a multidimensional data representation 1435 of the engine relative to a model (such as a CAD model) using tools (such as a head-mounted display). The inspection user interface 1410 may also facilitate remote determination of the remote arrangement of the engine or its components and remote supervision of the inspection 1440.

[0207] The inspection user interface 1410 facilitates multi-level data classification 1445 for analysis and report generation. Inspection data from image overlays 1425, virtual or augmented reality environment views 1430, multidimensional data representations 1435, and remote layout determination and inspection supervision 1440 can be classified and categorized for analysis and report generation. Inspection data can be categorized into one or more levels (e.g., assets, components, MRO, customers, regions) for analysis and report generation.

[0208] Multi-level data classification 1445 can classify inspection data related to Enterprise Resource Planning (ERP) to support processes in areas such as finance, human resources, manufacturing, supply chain, and procurement. Inspection data can be classified for ERP and merged into reports automatically transmitted to the ERP system 1450. For example, reports can be customized based on engineering costs, analysis-based maintenance (ABM) costs, and / or fleet management costs. Inspection data or reports summarizing inspection data can be transmitted to one or more users or systems associated with engineering, ABM, and / or fleet management.

[0209] Multi-level data classification 1445 can classify inspection data for MRO. Inspection data can be classified for MRO and presented to the user via the MRO user interface 1455. The MRO user interface facilitates the handling of MRO tasks. The MRO user interface can provide a work scope that describes the prescribed repairs and methods for repairing or maintaining the engine or components to meet repair or maintenance specifications based on the inspection data. The MRO user interface 1455 can define one or more MRO tasks to be performed based on the inspection data displayed.

[0210] Multi-level data classification 1445 can also classify inspection data for logistics optimization 1460. For example, during inspection, data regarding one or more problems or defects can be identified. Based on captured images and data acquired during inspection, the inspection computer system 110 can automatically identify repair or maintenance activities to resolve the problems or defects. These repair or maintenance activities can be compiled into logistics reports or can be automatically communicated to the logistics system for logistics optimization. For example, identified repair or maintenance activities may automatically trigger material ordering or arrange maintenance personnel for logistics purposes.

[0211] The multi-level data classification 1445 can also classify data for the customer user interface 1465. The customer user interface 1465 can provide asset conditions identified as a result of the inspection process. The customer user interface 1465 can also provide work processes that define the scope of tasks to be performed on the asset based on conditions or other issues identified during the inspection process. For example, the customer user interface 1465 can display a report detailing the maintenance task schedule to maintain the asset based on the asset conditions determined during the inspection.

[0212] Multi-level data classification 1445 can further classify data for supplier and / or reseller user interfaces 1470. As discussed above, based on captured images and data acquired during inspection, inspection computer system 110 can automatically identify repair or maintenance activities to resolve problems or defects. These repair or maintenance activities can be compiled into logistics reports or can be automatically communicated to logistics systems for logistics optimization. For example, identified repair or maintenance activities may automatically place orders with suppliers or resellers to procure the equipment or materials needed to perform the identified repair or maintenance.

[0213] Now go to Figures 15A-15C An exemplary inspection user interface is shown. In some embodiments, Figures 15A-15C The user interface shown can be accessed via a reference. Figure 1 The user interface device 140 described is shown.

[0214] Figure 15A This is the inspection user interface that allows the selection of new inspections. The inspection user interface presents different types of inspections to the inspection operator. Inspection types can involve different inspection recipes. The inspection operator can select inspections via the inspection user interface.

[0215] Figure 15B This is an inspection user interface that provides captured images from the inspection. Parts in the captured images are selected and outlined on the user interface, and the parts are identified on the user interface. The user interface provides options for selecting the arrangement of parts (e.g., installed, missing, damaged, unsuitable). The user interface also includes sections for users to enter comments or annotations on the parts.

[0216] Figure 15C This illustrates the inspection user interface that displays captured images during the inspection process. The captured images are tagged with metadata indicating the date and time they were captured. The user interface can be part of an inspection portal that provides inspection results and searchable images.

[0217] Figure 16 A head-mounted display 1700 is shown. The head-mounted display 1700 can be used as a user interface device 140. The head-mounted display 1700 includes a sensor assembly 1704 configured to serve as sensory input for receiving real-world information. The sensor assembly 1704 may include a camera, a GPS tracker, an accelerometer, and / or a gyroscope. The sensor assembly 1704 can capture images and track the user's movement and position. The sensor assembly also includes a near-eye display 1706 configured to present visual data close to the user's eyes. The head-mounted display also includes an arm 1702 extending rearward from the near-eye display 1706 and configured to be mounted on the user's head.

[0218] Figure 17It is an exemplary user interface device including screen 1705. In some embodiments, Figure 17 The user interface device shown may be a reference Figure 1 The user interface device 140 is described. Screen 1705 displays images from a sensor system 1710 coupled to a positioning system and an engine 1720 to be inspected. In some embodiments, the user interface device may include physical control elements such as a joystick or column 1730, a slider 1740, and a keypad 1750. Operation of the control column 1730 and slider 1740 can control the movement of the positioning system and / or control the display / playback of images on screen 1705. Screen 1705 may also display images captured by the sensor system 1710.

[0219] Further aspects of this disclosure are provided by the subject matter of the following clauses:

[0220] An aircraft component inspection system includes: an image capture system comprising: an image sensor system; a positioning system; and a processor communicatively coupled to the image sensor system and the positioning system, the processor being configured to: determine an inspection recipe based at least on an identifier associated with a component of the aircraft being inspected; identify multiple locations for image capture during an inspection workflow based on the inspection recipe; provide machine instructions to the positioning system to position the image sensor system relative to the component based on the multiple locations; cause the image sensor system to capture images at the multiple locations; and store the images having the captured locations in an inspection data database.

[0221] According to any of the foregoing clauses, the inspection recipe specifies an indication image capture location associated with at least one desired image, wherein the indication image capture location includes a coordinate position relative to the inspection space, the component, or a part of the component.

[0222] According to any of the foregoing clauses, the inspection formula specifies an indication of image capture orientation, including the roll, pitch, and / or yaw of the sensors of the image sensor system.

[0223] According to any of the foregoing clauses, the inspection recipe specifies a view of the component for at least one desired image, wherein the view of the component defines the size and / or orientation of a portion of the component within an image frame.

[0224] In any of the foregoing clauses, the processor is further configured to provide capture instructions to the image capture system based on the location of the image capture system or a portion thereof.

[0225] According to any of the foregoing clauses, the positioning system includes a snake-arm robot, and the machine instructions include instructions for controlling the movement of the snake-arm robot.

[0226] According to any of the foregoing clauses, the processor is configured to verify the captured image before storing the captured image in the inspection data database.

[0227] According to any of the foregoing clauses, the captured image is verified by the processor based on the location of the image capture system and / or the image quality.

[0228] According to any of the foregoing clauses, the inspection recipe identifies a portion of the component associated with the desired image, and the captured image is verified by the processor based on comparing the captured image with a computer model of the component, previously captured images from the component, and / or previously captured images from similar components associated with the portion of the component.

[0229] According to any of the foregoing clauses, the processor is further configured to: identify a plurality of parts of the component based on the captured images; automatically determine the condition of the parts based on at least one of the captured images; and determine subsequent tasks based on the condition of the parts.

[0230] According to any of the foregoing clauses, the system uses a machine learning model to identify the conditions of the part.

[0231] According to any of the foregoing clauses, the processor is further configured to determine the position and orientation of the component, and wherein the plurality of positions are further identified based on the position and orientation of the component.

[0232] According to any of the foregoing clauses, the position and orientation of the component are determined based on markings on or near the component, wherein the markings include optical markings, color-coded markings, shape-coded markings, pattern-coded markings, embossed markings, engraved markings, sonar-readable markings, and / or lidar-readable markings.

[0233] According to any of the foregoing clauses, the processor is further configured to determine an initial movement mode of the positioning system based on the plurality of locations.

[0234] According to any of the foregoing clauses, the processor is configured to determine an image quality metric of the image; determine an adjusted capture configuration based on the image quality metric; and instruct the image capture system of the adjusted capture configuration to recapture at least one of the images.

[0235] In any of the foregoing clauses, the machine instructions are determined by the processor based on a computer model simulating the movement of the positioning system using the components.

[0236] According to any of the foregoing clauses, the inspection recipe and / or the adaptation task of the inspection workflow is determined by the processor based on engine identifier, engine inspection history, engine construction record, engine repair history, customer-specified task requirements and / or identified problems.

[0237] According to any of the foregoing clauses of the system, the new capture location is determined by the processor based on the estimated location based on one or more defined regions of interest in the image.

[0238] The image capture system according to any of the foregoing clauses further includes: a user interface device; wherein the processor is further configured to: provide an inspection user interface on the user interface device; receive user input for an inspection task via the inspection user interface; and determine and communicate machine instructions for the positioning system and one or more sensors of the image capture system based on the user input.

[0239] According to any of the foregoing clauses, the user input includes touch input received on a touch-sensitive display showing an image of the component; wherein the processor is further configured to: determine one or more capture positions based on the touch input; determine movement of one or more sensors of the image capture system based on the one or more capture positions; and communicate machine instructions to the image capture system to cause the one or more sensors of the image capture system to move to the one or more capture positions.

[0240] An aircraft component inspection system includes: an image capture system; and a processor communicatively coupled to the image capture system, the processor being configured to: determine an inspection recipe based at least on an identifier associated with the aircraft component being inspected, the inspection recipe identifying a plurality of desired images to be captured during an inspection workflow; determine instructions for capturing at least one of the plurality of desired images identified in the inspection recipe; communicate the instructions to the image capture system to perform the inspection task via the image capture system; receive the captured images from the image capture system in response to communicating the instructions; and append metadata to the captured images and store the captured images in the engine history of the engine in an inspection database.

[0241] An aircraft component inspection system includes: an image capture system; and a processor communicatively coupled to the image capture system, the processor being configured to: receive one or more captured images from the image capture system; identify multiple components of an engine based on the one or more captured images; automatically determine conditions of the components based on the one or more captured images; and determine subsequent tasks based on the conditions associated with the multiple components of the engine.

[0242] An aircraft component inspection system includes: an image sensor system; a positioning system; and a processor communicatively coupled to the image sensor system and the positioning system. The processor is configured to: determine an inspection recipe based at least on identifiers associated with components of the aircraft being inspected; identify multiple locations for image capture during an inspection workflow based on the inspection recipe; provide machine instructions to the positioning system to position the image sensor system relative to the component based on the multiple locations; cause the image sensor system to capture images at the multiple locations; and store the images having capture location data in an inspection data database.

[0243] An aircraft component inspection system includes: an image capture system; and a processor communicatively coupled to the image capture system, the processor being configured to: determine an initial operating range of the image capture system based on an inspection recipe; communicate instructions to the image capture system based on the initial operating range; receive captured images of aircraft components from the image capture system; identify a task trigger based on the captured images; determine an adaptive capture task based on the task trigger; and communicate updated instructions to the image capture system based on the adaptive capture task during the operation of the operating range.

[0244] An aircraft component inspection system includes: an image capture system including a positioning system and a sensor system configured to capture images of the aircraft component; a user interface device; and a processor communicatively connected to the image capture system and the user interface device, and configured to: provide an inspection user interface on the user interface device; receive user input for an inspection task via the inspection user interface; and generate machine instructions for the positioning system and the sensor system based on the user input.

[0245] An aircraft component inspection system includes: an image capture system; and a processor communicatively coupled to the image capture system and configured to: receive captured images of components having additional metadata from the image capture system; retrieve supplementary information from a database based on the metadata associated with the images; provide an inspection user interface for display on a user interface device; and display the images together with the supplementary information in the inspection user interface.

[0246] An aircraft component inspection system includes: an image capture system; and a processor communicatively coupled to the image capture system and configured to: communicate a capture command to the image capture system to capture images of aircraft components for inspection; receive the captured images from the image capture system in response to the communication of the capture command; and store the captured images in an inspection database.

[0247] The system according to any of the foregoing clauses, wherein the component includes an aircraft engine.

[0248] According to any of the foregoing clauses, the processor is further configured to: determine an inspection recipe based at least on an identifier associated with the component, wherein the inspection recipe identifies a plurality of desired images to be captured during the inspection work; and receive captured images from the image capture system in response to a communication instruction; and store the captured images in the component history of the component in an inspection database; wherein the capture instruction is determined based on the plurality of desired images.

[0249] According to any of the foregoing clauses, the inspection recipe is retrieved from a recipe database based on the identifier associated with the component.

[0250] According to any of the foregoing clauses, the inspection recipe is generated based on a recipe machine learning model trained on multiple image sets captured from multiple parts, physical part models and / or computer part models.

[0251] According to any of the foregoing clauses, the inspection recipe is generated based on mapping a reference image from a maintenance manual associated with the component to a computer model of the component using a computer vision algorithm.

[0252] According to any of the foregoing clauses, the inspection recipe is generated based on simulating camera views on a computer model of the component to define a minimum set of views covering a predefined portion of the component.

[0253] According to any of the foregoing clauses, the inspection formula is determined based on component inspection history, component repair history, customer-specified task requirements, and / or identified problems.

[0254] According to any of the foregoing clauses, the inspection recipe specifies an indication image capture location associated with the at least one desired image, wherein the indication capture location includes a coordinate position relative to the inspection space, the component, or a part of the component.

[0255] According to any of the foregoing clauses, the inspection formula specifies an indication image capture location associated with the at least one desired image, wherein the indication capture location includes the distance from the part of the component.

[0256] According to any of the foregoing clauses, the inspection formula specifies an indication of image capture orientation, including the roll, pitch, and / or yaw of the sensors of the image capture system.

[0257] According to any of the foregoing clauses, the inspection recipe specifies a view of the component for the at least one desired image, wherein the view of the component defines the size and / or orientation of a portion of the component within an image frame.

[0258] According to any of the foregoing clauses, the inspection formula specifies the image type and / or image capture system type of the at least one desired image.

[0259] The system according to any of the foregoing clauses, wherein the image capture system includes one or more of an optical sensor, an infrared sensor, a terahertz spectrometer, a microwave imaging sensor, an X-ray imager, a computed tomography scanner, an eddy current imaging sensor, or an ultrasound imager.

[0260] The system according to any of the foregoing clauses, wherein the image capture system includes a user interface device for displaying the instructions to a user.

[0261] According to any of the foregoing clauses, the instructions include an augmented reality or mixed reality display shown on the user interface device, the augmented reality or mixed reality display overlaying the instructions onto a view of a portion of the component.

[0262] According to any of the foregoing clauses, the instructions displayed on the user interface device include a reference image of a portion of the component to be captured, an outline of the portion of the component to be captured, and / or an image of the component having an identifier marking the location of the portion of the component to be captured.

[0263] According to any of the foregoing clauses, the processor is further configured to: identify the position of the image capture system relative to the component based on a position sensor on the image capture system, an image captured by the image capture system, or an image captured by a separate sensor system.

[0264] In any of the foregoing clauses, the processor is further configured to provide capture instructions to the image capture system based on the location of the image capture system.

[0265] According to any of the foregoing clauses, the processor is further configured to: determine an image capture system orientation, the capture instructions being further provided based on the image capture system orientation.

[0266] According to any of the foregoing clauses, the instructions are configured to automatically set an image capture configuration on the image capture system, wherein the image capture configuration includes zoom level, focal length, exposure time, light sensitivity setting, illumination setting, image resolution, video length, and / or sensor type selection.

[0267] The system according to any of the foregoing clauses, wherein the image capture system includes a mobile computer, mobile phone, tablet computer, or head-mounted display device.

[0268] The system according to any of the foregoing clauses further includes a projection display device configured to project instructions from the instruction set onto the surface of the component being inspected and / or around the component being inspected.

[0269] The system according to any of the foregoing clauses, wherein the image capture system includes an autonomous ground vehicle, a robotic arm, a snake-arm robot, and / or an orbital camera system, and the instructions include machine instructions for controlling the movement of the image capture system.

[0270] The system according to any of the foregoing clauses further includes a second image capture system, and the processor is further configured to simultaneously provide instructions to the image capture system and the second image capture system based on the recipe to capture the desired image specified in the recipe.

[0271] According to any of the foregoing clauses, the processor is configured to verify the captured image before storing the captured image in the inspection database.

[0272] According to any of the foregoing clauses, the captured image is verified based on the image capture system location and / or image quality.

[0273] According to any of the foregoing clauses, the inspection formula identifies a portion of the component associated with the desired image and verifies the captured image based on the detection of the portion of the component in the captured image, machine object detection, machine feature detection, object recognition algorithm and / or optical character recognition algorithm.

[0274] According to any of the foregoing clauses, the inspection recipe identifies a portion of the component associated with the desired image and verifies the captured image by comparing the captured image with a computer model of the component, previously captured images from the component, and / or previously captured images from similar components associated with the portion of the component.

[0275] According to any of the foregoing clauses, the inspection formula identifies a computer model of the component and verifies the captured image based on identifying gaps in the coverage area by comparing the computer model with the captured image; wherein the processor is further configured to instruct additional capture tasks based on the gaps in the coverage area.

[0276] According to any of the foregoing clauses, in the event that the captured image fails verification, the processor is further configured to send an updated instruction to the image capture system instructing the recapture of the desired image.

[0277] According to any of the foregoing clauses, the processor is further configured to verify the inspection by comparing portions of the component imaged by images captured during the inspection work period with the integrity requirements specified in the inspection recipe.

[0278] According to any of the foregoing clauses, the processor is further configured to: in response to receiving instructions from the processor, determine the scope of work for a repair or maintenance task based on a plurality of images captured by the image capture system.

[0279] According to any of the foregoing clauses, the processor is further configured to: identify anomalies based on data recorded by the image capture system before completing the execution of the inspection scope; and modify instructions communicated to the image capture system based on the anomalies.

[0280] According to any of the foregoing clauses, the processor is further configured to append metadata to the captured image in the inspection database.

[0281] According to any of the foregoing clauses, the metadata includes image capture system location, image capture system orientation, image capture system identifier, imaging component location, component identifier, and / or timestamp.

[0282] According to any of the foregoing clauses, the inspection recipe associates the desired image with a component identifier, and the metadata includes the component identifier.

[0283] According to any of the foregoing clauses, the metadata includes component identifiers determined based on machine object detection, machine feature detection, object recognition algorithms and / or optical character recognition algorithms.

[0284] According to any of the foregoing clauses, the metadata is overlaid on the captured image.

[0285] According to any of the foregoing clauses, the processor is further configured to: identify multiple parts of the component based on the captured images; automatically determine the conditions of the parts based on at least one of the captured images; and determine subsequent tasks based on the conditions associated with the parts.

[0286] According to any of the foregoing clauses, the system uses a part recognition machine learning model to identify the plurality of parts.

[0287] According to any of the foregoing clauses, the processor is further configured to train the part recognition machine learning model using multiple images of the parts and part identifiers associated with each part.

[0288] According to any of the foregoing clauses, the processor is further configured to: receive operator feedback regarding the identification of the plurality of parts, and further train the part identification machine learning model based on the feedback.

[0289] According to any of the foregoing clauses, the plurality of parts are identified based on data cards associated with the plurality of parts identified in one or more of the captured images.

[0290] According to any of the foregoing clauses, the plurality of parts are identified based on an optical character recognition algorithm performed on one or more of the captured images.

[0291] According to any of the foregoing clauses, the plurality of parts are identified based on the shape, color, and / or position of the parts appearing in one or more of the captured images.

[0292] According to any of the foregoing clauses, the condition of the part is identified based on the presence and / or anomalies of the part detected in one or more images of the part.

[0293] According to any of the foregoing clauses, the system uses a part condition machine learning model to identify the condition of the part.

[0294] According to any of the foregoing clauses, the processor is configured to train a part-conditional machine learning model using a training set comprising multiple images of one or more parts labeled with associated conditions.

[0295] According to any of the foregoing clauses, the processor is further configured to train the part conditional machine learning model using multiple images of the part and conditional identifiers associated with each image.

[0296] According to any of the foregoing clauses, the part-conditional machine learning model is further trained on background data associated with each image, wherein the background data includes customer data, geographic data, route data, assembly data, and service history data.

[0297] According to any of the foregoing clauses, the processor is further configured to: receive operator feedback regarding the identification of the conditions of the part, and further train a machine learning model of the part conditions based on the feedback.

[0298] According to any of the foregoing clauses of the system, the conditions of the part include the presence or absence of the part, the maintainability of the part, the task capability of the part, and / or the maintenance or repair task of the part.

[0299] According to any of the foregoing clauses, in response to the condition that the part is unavailable or unusable, the processor is configured to forward the procurement request to the logistics system.

[0300] According to any of the foregoing clauses, the processor is further configured to compare the plurality of parts with records in an asset tracking database to determine the authenticity, manufacturer, or origin of the plurality of parts.

[0301] According to any of the foregoing clauses, the processor is further configured to compare the plurality of parts with a set of manufacturing standards to determine the standard compliance of the plurality of parts.

[0302] According to any of the foregoing clauses, the processor is further configured to compare the conditions of one or more parts with a task requirement profile to determine a task suitability matrix for the parts.

[0303] According to any of the foregoing clauses, the processor is further configured to compare the condition of one or more parts with market value reference data to determine the market value of the part or one or more parts of the part.

[0304] According to any of the foregoing clauses of the system, the subsequent tasks include one or more replacement tasks, disassembly tasks, repair tasks, cleaning tasks and / or further inspection tasks of one or more of the plurality of parts.

[0305] According to any of the foregoing clauses, the processor is further configured to determine the authorization status of the subsequent task based on the conditions of the plurality of components and customer-specified requirements.

[0306] According to any of the foregoing clauses of the system, the authorization status of the subsequent task is further determined based on lifetime and durability data associated with the component.

[0307] According to any of the foregoing clauses, the image capture system includes an image sensor system and a positioning system; and the processor is configured to: identify multiple locations for image capture during the inspection work range based on the inspection recipe; wherein the capture instructions include: machine instructions of the positioning system for positioning one or more sensors of the image sensor system relative to the component; and machine instructions of the one or more sensors of the image sensor system for capturing an image.

[0308] According to any of the foregoing clauses, the positioning system includes a gantry configured to move one or more sensors of the image sensor system around the component.

[0309] According to any of the foregoing clauses, the positioning system includes a plurality of gantry frames, each gantry frame being coupled to one or more sensors of the image sensor system to form an imaging gantry.

[0310] According to any of the foregoing clauses, in a system wherein one or more sensors of the image sensor system are mounted on a gantry, the gantry being configured to move the one or more sensors in a first plane, and a track system being configured to move the gantry in a direction perpendicular to the first plane.

[0311] The system according to any of the foregoing clauses, wherein the positioning system includes a powered vehicle carrying one or more sensors of the image sensor system.

[0312] According to any of the foregoing clauses, the positioning system further includes a track defining the path of the powered vehicle.

[0313] The system according to any of the foregoing clauses, wherein the powered vehicle includes a drone.

[0314] The system according to any of the foregoing clauses, wherein the positioning system includes a cable-suspended camera positioning system.

[0315] The system according to any of the foregoing clauses, wherein the positioning system includes a programmable robotic arm.

[0316] According to any of the foregoing clauses, the positioning system is mounted on a bracket supporting the component.

[0317] The system according to any of the foregoing clauses, wherein the positioning system comprises two or more of a gantry system, track system, powered vehicle, component turntable or cable-suspended camera positioning system configured to simultaneously move different sensors of the image sensor system to perform the inspection.

[0318] According to any of the foregoing clauses, the positioning system is configured to move the component relative to one or more stationary sensors of the image sensor system.

[0319] According to any of the foregoing clauses, the positioning system includes a turntable configured to rotate the component on an axis.

[0320] The system according to any of the foregoing clauses, wherein the image sensor system includes one or more of an optical sensor, LIDAR, 3D scanner, infrared sensor, terahertz spectrometer, microwave imaging sensor, X-ray imager, computed tomography scanner, eddy current imaging sensor or ultrasound imager.

[0321] The system according to any of the foregoing clauses further includes a lighting system, and the processor is configured to provide illumination instructions to the lighting system based on the configuration.

[0322] According to any of the foregoing clauses, the inspection recipe specifies an indication image capture location associated with at least one desired image, wherein the indication capture location includes a coordinate position relative to the inspection space, the component, or a part of the component.

[0323] According to any of the foregoing clauses, the inspection formula specifies an indication of image capture orientation, including the roll, pitch, and / or yaw of the sensors of the image capture system.

[0324] According to any of the foregoing clauses, the processor is further configured to determine the position and orientation of the component, and the plurality of positions are further identified based on the position and orientation of the component.

[0325] According to any of the foregoing clauses, the position and orientation of the component are determined by markings on a support that supports the component.

[0326] According to any of the foregoing clauses, the position and orientation of the component are determined based on markings on or near the component, wherein the markings include optical markings, color-coded markings, shape-coded markings, pattern-coded markings, embossed markings, engraved markings, sonar-readable markings, and / or lidar-readable markings.

[0327] According to any of the foregoing clauses, the plurality of locations includes sensor locations for locating one or more sensors of the image sensor system within the imaging space.

[0328] According to any of the foregoing clauses, the processor is further configured to select at least one sensor of the image sensor system for each of the plurality of locations based on the inspection formula.

[0329] According to any of the foregoing clauses, the processor is further configured to determine an image capture configuration for each of the plurality of locations based on the inspection recipe, and to provide capture configuration instructions to the image sensor system based on the image capture configuration.

[0330] According to any of the foregoing clauses, the image capture configuration includes zoom level, focal length, aperture, exposure time, light sensitivity setting, illumination setting, image resolution, video length, and / or sensor type selection.

[0331] According to any of the foregoing clauses, the processor is further configured to determine an initial movement mode of the positioning system based on the plurality of locations.

[0332] According to any of the foregoing clauses, the processor is further configured to select additional capture locations based on images captured by the image sensor system, and to provide updated machine instructions to the positioning system based on the additional capture locations.

[0333] According to any of the foregoing clauses, the processor is further configured to verify the captured image based on the image requirements in the inspection recipe.

[0334] According to any of the foregoing clauses, the image requirements include focus, focus of the point of interest, blur amount, exposure, brightness, overlap with adjacent images, identification of parts of the component in the overlapping portion, and the presence of parts of the component.

[0335] According to any of the foregoing clauses, in a system wherein, if the captured image does not meet the image requirements, the processor is configured to select a revised capture location and / or a revised capture configuration, and to provide machine instructions to the positioning system and / or the image sensor system based on the revised capture location and / or the revised capture configuration.

[0336] According to any of the foregoing clauses, the processor is configured to: determine an image quality metric of the captured image; determine an adjusted capture configuration based on the image quality metric; and instruct the image capture system to recapture the image.

[0337] According to any of the foregoing clauses, the adjusted capture configuration is determined based on a machine learning model trained with an image quality training dataset, which includes multiple captured images, each labeled with a capture location, capture configuration, and quality metric.

[0338] According to any of the foregoing clauses, the machine instructions of the positioning system are determined based on simulating the movement of the positioning system using a computer model of the components.

[0339] According to any of the foregoing clauses, the processor is configured to determine the capture order of the plurality of locations and, based on the capture order, determine the machine instructions of the positioning system.

[0340] According to any of the foregoing clauses, the processor is further configured to: determine an initial operating range of the image capture system based on a check recipe; communicate instructions to the image capture system based on the initial operating range; recognize trigger conditions based on the captured images; determine an adaptation task based on the trigger conditions; and communicate updated instructions to the image capture system based on the adaptation task during the execution of the initial operating range.

[0341] According to any of the foregoing clauses, the inspection recipe includes image data requirements; and the triggering condition is identified based on comparing the captured image with the image data requirements of the inspection recipe.

[0342] According to any of the foregoing clauses, the inspection formula identifies the part of the component and verifies the captured image based on comparing the captured image with a computer model of the part, previously captured images from the part, and / or previously captured images from similar parts.

[0343] According to any of the foregoing clauses, the inspection recipe is generated based on a machine learning model trained on multiple image sets captured from multiple engines.

[0344] According to any of the foregoing clauses, the inspection recipe is generated based on mapping a reference image from a maintenance manual associated with the component to a computer model of the component using a computer vision algorithm.

[0345] According to any of the foregoing clauses, the inspection formula and / or the adaptation task are determined based on the engine identifier, engine inspection history, engine construction record, engine repair history, customer-specified task requirements, and / or identified problems.

[0346] According to any of the foregoing clauses, the inspection recipe is determined based on a recipe machine learning model trained on multiple images, the multiple images including labeled image quality metrics, component identifiers, component repair history, geographic regions associated with the component, component flight path history, and / or component operator identifiers.

[0347] According to any of the foregoing clauses, the plurality of images include images of other components of the same or similar type, images of component models, and computer simulations of said components.

[0348] According to any of the foregoing clauses, the processor is further configured to verify the image quality of the captured image, and the triggering condition includes the captured image failing the verification.

[0349] According to any of the foregoing clauses, the image quality includes image resolution, illumination, sharpness, and / or the framing of the region of interest.

[0350] According to any of the foregoing clauses, the captured image is verified based on whether a specified part of the component identified in the inspection recipe is visible in the captured image, and whether the specified part of the component identified in the inspection recipe is visible in the captured image is determined based on reference image comparison, machine object detection, machine feature defects, object recognition algorithm and / or optical character recognition algorithm.

[0351] According to any of the foregoing clauses, the triggering conditions include the detection of anomalies and / or regions of interest in the captured image.

[0352] According to any of the foregoing clauses, the triggering condition is detected based on conditional evaluation of the component using a machine learning model trained on a sample inspection image and conditions associated with the sample inspection image.

[0353] According to any of the foregoing clauses, the triggering conditions are further detected based on component identifiers, component repair history, geographic regions associated with component use, component flight path history, and / or component operator identifiers.

[0354] According to any of the foregoing clauses, the triggering condition is detected based on multiple captured images.

[0355] According to any of the foregoing clauses, the system uses a conditional machine learning model trained on multiple images labeled with trigger conditions to detect the trigger conditions.

[0356] According to any of the foregoing clauses, the adaptation task includes an image capture task at a new capture location and / or with a new capture configuration.

[0357] According to any of the foregoing clauses, the new capture configuration includes zoom level, exposure time, light sensitivity setting, illumination setting, image resolution, video length, and / or sensor type selection.

[0358] According to any of the foregoing clauses, the new capture location is determined based on the estimated location of a region of interest determined based on one or more captured images.

[0359] According to any of the foregoing clauses, the new capture position is determined based on the movement of the positioning system of the image capture system simulated using a computer model of the components.

[0360] According to any of the foregoing clauses, the instructions for movement of the positioning system are simulated using a machine learning model trained on real-world and / or simulated positioning system movement.

[0361] According to any of the preceding clauses, the new capture location and / or the new capture configuration is determined based on an image capture machine learning model trained on a training dataset comprising multiple images labeled with image quality metrics, capture locations, and capture configurations.

[0362] According to any of the foregoing clauses, the new capture configuration is determined based on image quality analysis using one or more captured images.

[0363] According to any of the foregoing clauses, the adaptation task includes a detailed query task for the identified region of interest.

[0364] According to any of the foregoing clauses, the adaptation task includes the task of capturing multiple images with different capture configurations.

[0365] According to any of the foregoing clauses, the adaptation task includes the task of capturing multiple images at multiple additional capture locations and / or at multiple additional capture angles.

[0366] According to any of the foregoing clauses, the adaptation task includes a set of predefined capture locations around the identified region of interest.

[0367] According to any of the foregoing clauses, the adaptation task includes repair or maintenance tasks.

[0368] According to any of the foregoing clauses, the captured image is captured by a first sensor of the image capture system, and an adaptive capture command is configured to cause a second sensor of the image capture system to capture the image.

[0369] According to any of the foregoing clauses, the captured image is captured using a first capture configuration, and the adaptive capture instruction is configured to cause the image capture system to capture the image using a second capture configuration.

[0370] According to any of the foregoing clauses of the system, the updated instructions are determined based on a localization system motion machine learning model, which is configured to determine a path from a first capture position of the captured image to a second capture position of the adaptation task.

[0371] According to any of the foregoing clauses, the processor is further configured to: provide a reviewer user interface on a user interface device; transmit the captured image and / or the adaptation task to the reviewer user interface for display; and determine instructions for the adaptation task based on user input received via the reviewer user interface.

[0372] The system according to any of the foregoing clauses further includes: a user interface device; wherein the image capture system includes a positioning system and a sensor system configured to capture images of the components of the aircraft; wherein the processor is configured to: provide an inspection user interface on the user interface device; receive user input for an inspection task via the inspection user interface; and determine and communicate machine instructions for one or more sensors of the positioning system and the sensor capture system based on the user input.

[0373] According to any of the foregoing clauses, the inspection user interface includes controls for a plurality of sensors of the sensor system and a plurality of positioning devices of the positioning system, the controls being selectively and simultaneously displayed on the display of the user interface device.

[0374] According to any of the foregoing clauses, the inspection user interface includes a representation of the components of the aircraft and a representation of the sensors of the sensor system, the representation of the sensors being draggable and positionable around the representation of the components.

[0375] According to any of the foregoing clauses, the positioning system includes an engine rotation device configured to rotate parts within the engine.

[0376] In some respects, the technology described herein relates to the system in which the components include turbine blades, disks, bladed disks, and / or shafts.

[0377] According to any of the foregoing clauses, the inspection user interface includes a first slider for controlling the rotation of the engine rotation device.

[0378] According to any of the foregoing clauses, the inspection user interface further includes an image of the engine for selecting the insertion position of the endoscope.

[0379] According to any of the foregoing clauses, the processor is configured to determine the acquisition timing of the sensor system based on the movement of the positioning system, and to provide the machine instructions to the sensor system based on the acquisition timing.

[0380] According to any of the foregoing clauses, the positioning system includes a component positioning system and a sensor positioning system, and the processor is configured to simultaneously control the movement of the component positioning system and the sensor positioning system to affect the relative position of the sensor system and the component used for image capture.

[0381] The system according to any of the foregoing clauses, wherein the user interface device includes a head-mounted display.

[0382] According to any of the foregoing clauses, the user input includes movement and orientation of the head-mounted display, and the processor is configured to control movement of the image capture device via the positioning system based on the movement and orientation of the head-mounted display.

[0383] According to any of the foregoing clauses, the user input includes touch input received on a touch-sensitive display showing an image of the component; wherein the processor is configured to: determine one or more capture positions based on the touch input; determine movement of one or more sensors of the image capture system based on the one or more capture positions; and communicate machine instructions to the image capture system to cause one or more sensors of the image capture system to move to the one or more capture positions.

[0384] According to any of the foregoing clauses, the touch input includes tapping, dragging, and / or multi-touch squeezing or stretching actions.

[0385] According to any of the foregoing clauses, the dragging action corresponds to movement in a plane parallel to the image of the component, and the squeezing or stretching action corresponds to forward or backward movement relative to the plane.

[0386] According to any of the foregoing clauses, the processor is configured to: display a previously captured image or model of the component; receive a user selection of a region of interest in the previously captured image as the user input; and determine a capture location and / or capture configuration based on the region of interest.

[0387] According to any of the foregoing clauses, the capture location and the capture configuration are selected to focus and magnify the region of interest.

[0388] The system according to any of the foregoing clauses, wherein the inspection user interface includes a virtual, augmented, or mixed reality display.

[0389] In any of the foregoing clauses, the controls of the image capture system are displayed as an overlay on the view of the sensor system.

[0390] In any of the foregoing clauses, the user input includes motion input captured by a motion sensor.

[0391] According to any of the foregoing clauses, the user interface device includes a control column or control lever for controlling the movement of the positioning system and / or for reviewing images captured by the image capture system.

[0392] According to any of the foregoing clauses, the processor is further configured to: receive a captured image with additional metadata from the image capture system; and selectively display a subset of available controls of the positioning system and the sensor system based on the metadata of the captured image.

[0393] According to any of the foregoing clauses, a subset of the available controls is selected based on the location and / or part identifier indicated in the metadata of the captured image.

[0394] According to any of the foregoing clauses, the processor is further configured to: retrieve a captured image of a component captured by the image capture system and metadata associated with the captured image; retrieve supplementary information from a database based on the metadata associated with the image; provide an inspection user interface for display on a user interface device; and display the image together with the supplementary information on the inspection user interface.

[0395] According to any of the foregoing clauses, the system wherein the metadata identifies a view or part of the component, and the supplementary information includes technical or maintenance manual data associated with the part of the component.

[0396] According to any of the foregoing clauses, the system wherein the metadata identifies a problem with a part of the component, and the supplementary information includes repair options associated with the problem.

[0397] According to any of the foregoing clauses, the processor is further configured to: identify a role associated with a user of the user interface device; retrieve a report template from a report template database based on the role associated with the user; and selectively populate the report template with captured images and associated metadata stored in an inspection database.

[0398] According to any of the foregoing clauses, the system wherein the report template is generated by a report machine learning model trained on historical reports from multiple roles.

[0399] In any of the foregoing clauses, the roles are selected from inspector, administrator, customer, seller, and auditor roles.

[0400] An aircraft inspection method includes: determining an inspection recipe based at least on an identifier associated with an aircraft component being inspected, the inspection recipe identifying a plurality of desired images to be captured during an inspection workflow; determining instructions for capturing at least one desired image of the plurality of desired images identified in the inspection recipe; communicating the instructions to the image capture system to perform the inspection task via the image capture system; receiving the captured image from the image capture system in response to communicating the instructions; and appending metadata to the captured image and storing the captured image in the engine history of the engine in an inspection database.

[0401] An aircraft inspection method includes: receiving one or more captured images from the image capture system; identifying multiple components of an engine based on the one or more captured images; automatically determining the conditions of the components based on the one or more captured images; and determining subsequent tasks based on the conditions associated with the multiple components of the engine.

[0402] An aircraft inspection method includes: determining an inspection recipe based at least on identifiers associated with components of the aircraft being inspected; identifying multiple locations for image capture during an inspection workflow based on the inspection recipe; providing machine instructions to the positioning system to position an image sensor system relative to the components based on the multiple locations; causing the image sensor system to capture images at the multiple locations; and storing the images having capture location data in an inspection data database.

[0403] An aircraft inspection method includes: determining an initial operating range of an image capture system based on an inspection recipe; communicating instructions to the image capture system based on the initial operating range; receiving captured images of aircraft components from the image capture system; identifying a task trigger based on the captured images; determining an adaptive capture task based on the task trigger; and communicating updated instructions to the image capture system based on the adaptive capture task during the operation of the operating range.

[0404] An aircraft inspection method includes: providing an inspection user interface on the user interface device; receiving user input for an inspection task via the inspection user interface; and generating machine instructions for the positioning system and the sensor system based on the user input.

[0405] An aircraft inspection method includes: receiving a captured image of a component having additional metadata from the image capture system; retrieving supplementary information from a database based on the metadata associated with the image; providing an inspection user interface for display on a user interface device; and displaying the image together with the supplementary information on the inspection user interface.

[0406] An aircraft inspection method includes: communicating a capture command to an image capture system to capture images of aircraft components for inspection; receiving the captured images from the image capture system in response to communicating the capture command; and storing the captured images in an inspection database.

[0407] The method according to any of the foregoing clauses, wherein the component includes an aircraft engine.

[0408] The method according to any of the foregoing clauses further includes: determining an inspection recipe based at least on an identifier associated with the component, wherein the inspection recipe identifies a plurality of desired images to be captured during the inspection work scope; receiving captured images from the image capture system in response to a communication instruction; and storing the captured images in the component history of the component in an inspection database; wherein the capture instruction is determined based on the plurality of desired images.

[0409] According to the method described in any of the foregoing clauses, the inspection formula is retrieved from a formula database based on the identifier associated with the component.

[0410] According to any of the foregoing clauses, the inspection recipe is generated based on a recipe machine learning model trained on multiple image sets captured from multiple parts, physical part models and / or computer part models.

[0411] According to any of the foregoing clauses, the inspection recipe is generated based on mapping a reference image from a maintenance manual associated with the component to a computer model of the component using a computer vision algorithm.

[0412] According to any of the foregoing clauses, the method wherein the inspection formula is generated based on simulating camera views on a computer model of the component to define a minimum set of views covering a predefined portion of the component.

[0413] The method described in any of the foregoing clauses, wherein the inspection formula is determined based on component inspection history, component repair history, customer-specified task requirements, and / or identified problems.

[0414] According to any of the foregoing clauses, the inspection recipe specifies an indication image capture location associated with the at least one desired image, wherein the indication capture location includes a coordinate position relative to the inspection space, the component, or a part of the component.

[0415] According to any of the foregoing clauses of the method, wherein the inspection formula specifies an indication image capture location associated with the at least one desired image, wherein the indication capture location includes the distance from the part of the component.

[0416] According to any of the foregoing clauses of the method, wherein the inspection formula specifies an indication of image capture orientation, including the roll, pitch, and / or yaw of the sensors of the image capture system.

[0417] According to any of the foregoing clauses, the method wherein the inspection formula specifies a view of the component for the at least one desired image, wherein the view of the component defines the size and / or orientation of a portion of the component within an image frame.

[0418] According to the method described in any of the foregoing clauses, wherein the inspection formula specifies the image type and / or image capture system type of the at least one desired image.

[0419] The method according to any of the foregoing clauses, wherein the image capture system comprises one or more of an optical sensor, an infrared sensor, a terahertz spectrometer, a microwave imaging sensor, an X-ray imager, a computed tomography scanner, an eddy current imaging sensor, or an ultrasound imager.

[0420] The method according to any of the foregoing clauses, wherein the image capture system includes a user interface device for displaying the instructions to a user.

[0421] According to any of the foregoing clauses, the instructions include an augmented reality or mixed reality display shown on the user interface device, the augmented reality or mixed reality display overlaying the instructions onto a view of a portion of the component.

[0422] According to any of the foregoing clauses, the instructions displayed on the user interface device include a reference image of a portion of the component to be captured, an outline of the portion of the component to be captured, and / or an image of the component having an identifier marking the location of the portion of the component to be captured.

[0423] The method according to any of the foregoing clauses further includes: identifying the position of the image capture system relative to the component based on a position sensor on the image capture system, an image captured by the image capture system, or an image captured by a separate sensor system.

[0424] The method according to any of the foregoing clauses further includes: providing a capture command to the image capture system based on the location of the image capture system.

[0425] The method according to any of the foregoing clauses further includes: determining an image capture system orientation, wherein the capture instructions are further provided based on the image capture system orientation.

[0426] According to any of the foregoing clauses, the instructions are configured to automatically set an image capture configuration on the image capture system, and the image capture configuration includes zoom level, focal length, exposure time, light sensitivity setting, illumination setting, image resolution, video length, and / or sensor type selection.

[0427] The method according to any of the foregoing clauses, wherein the image capture system includes a mobile computer, mobile phone, tablet computer, or head-mounted display device.

[0428] The method according to any of the foregoing clauses further includes: projecting instructions from the instruction set onto the surface of the component being inspected and / or around the component being inspected using a constructed projection display device.

[0429] According to any of the foregoing clauses, the image capture system includes an autonomous ground vehicle, a robotic arm, a snake-arm robot, and / or a track-mounted camera system, and the instructions include machine instructions for controlling the movement of the image capture system.

[0430] The method according to any of the foregoing clauses further includes: simultaneously providing instructions to the image capture system and the second image capture system based on the recipe to capture the desired image specified in the recipe.

[0431] The method according to any of the foregoing clauses further includes: verifying the captured image before storing the captured image in the inspection database.

[0432] According to any of the foregoing clauses, the captured image is verified based on the image capture system location and / or image quality.

[0433] According to any of the foregoing clauses, the method of inspection identifies a portion of the component associated with the desired image and verifies the captured image based on the detection of the portion of the component in the captured image, machine object detection, machine feature detection, object recognition algorithm and / or optical character recognition algorithm.

[0434] According to any of the foregoing clauses, the method wherein the inspection formula identifies a portion of the component associated with the desired image and verifies the captured image by comparing the captured image with a computer model of the component, a previously captured image from the component, and / or a previously captured image from a similar component associated with the portion of the component.

[0435] According to any of the foregoing clauses, the inspection formula identifies a computer model of the component and verifies the captured image based on identifying gaps in the coverage area by comparing the computer model with the captured image; wherein the processor is further configured to instruct additional capture tasks based on the gaps in the coverage area.

[0436] According to any of the foregoing clauses of the method, wherein, in the event that the captured image fails verification, the processor is further configured to send an updated instruction to the image capture system instructing the recapture of the desired image.

[0437] The method according to any of the foregoing clauses further includes: verifying the inspection by comparing a portion of the component imaged by an image captured during the inspection work period with the integrity requirements specified in the inspection recipe.

[0438] The method according to any of the foregoing clauses further includes: in response to receiving instructions from the processor, determining the scope of a repair or maintenance task based on a plurality of images captured by the image capture system.

[0439] The method according to any of the foregoing clauses further includes: identifying anomalies based on data recorded by the image capture system before completing the execution of the inspection scope; and modifying the instructions communicated to the image capture system based on the anomalies.

[0440] The method according to any of the foregoing clauses further includes: appending metadata to the captured image in the inspection database.

[0441] According to any of the foregoing provisions of the method, the metadata includes image capture system location, image capture system orientation, image capture system identifier, imaging component location, component identifier and / or timestamp.

[0442] According to any of the foregoing clauses, the method wherein the inspection formula associates the desired image with a component identifier, and the metadata includes the component identifier.

[0443] According to any of the foregoing provisions, the metadata includes component identifiers determined based on machine object detection, machine feature detection, object recognition algorithms and / or optical character recognition algorithms.

[0444] According to any of the foregoing provisions, the metadata is overlaid on the captured image.

[0445] The method according to any of the foregoing clauses further includes: identifying multiple parts of the component based on the captured images; automatically determining the condition of the part based on at least one of the captured images; and determining subsequent tasks based on the condition associated with the part.

[0446] According to any of the foregoing clauses, the method wherein a part recognition machine learning model is used to identify the plurality of parts.

[0447] The method according to any of the foregoing clauses further includes: training the part recognition machine learning model using multiple images of the parts and part identifiers associated with each part.

[0448] The method according to any of the foregoing clauses further includes: receiving operator feedback regarding the identification of the plurality of parts, and further training the part identification machine learning model based on the feedback.

[0449] According to the method described in any of the foregoing clauses, the plurality of parts are identified based on identifying data cards associated with the plurality of parts in one or more of the captured images.

[0450] According to any of the foregoing provisions of the method, the plurality of parts are identified based on an optical character recognition algorithm performed on one or more of the captured images.

[0451] According to any of the foregoing clauses, the plurality of parts are identified based on the shape, color, and / or position of the parts appearing in one or more of the captured images.

[0452] According to any of the foregoing clauses of the method, the condition of the part is identified based on the presence and / or anomalies of the part detected in one or more images of the part.

[0453] According to any of the foregoing provisions, the condition of the part is identified using a part condition machine learning model.

[0454] The method according to any of the foregoing clauses further includes: training a part condition machine learning model using a training set, said training set comprising multiple images of one or more parts labeled with associated conditions.

[0455] The method according to any of the foregoing clauses further includes: training the part conditional machine learning model using multiple images of the part and conditional identifiers associated with each image.

[0456] According to any of the foregoing clauses, the part-conditional machine learning model is further trained on background data associated with each image, wherein the background data includes customer data, geographic data, route data, assembly data, and service history data.

[0457] The method according to any of the foregoing clauses further includes: receiving operator feedback regarding the identification of the condition of the part, and further training the part condition machine learning model based on the feedback.

[0458] According to any of the foregoing clauses of the method, the conditions of the part include the presence or absence of the part, the maintainability of the part, the task capability of the part, and / or the maintenance or repair tasks of the part.

[0459] According to any of the foregoing clauses, the processor is configured to forward a procurement request to the logistics system in response to the condition that the part is unavailable or unusable.

[0460] The method according to any of the foregoing clauses further includes: comparing the plurality of parts with records in an asset tracking database to determine the authenticity, manufacturer, or origin of the plurality of parts.

[0461] The method according to any of the foregoing clauses further includes: comparing the plurality of parts with a set of manufacturing standards to determine the standard compliance of the plurality of parts.

[0462] The method according to any of the foregoing clauses further includes: comparing the conditions of one or more parts with a task requirements profile to determine a task suitability matrix for the parts.

[0463] The method according to any of the foregoing clauses further includes: comparing the condition of one or more parts with market value reference data to determine the market value of the component or one or more parts of the component.

[0464] According to any of the foregoing clauses, the subsequent tasks include one or more replacement tasks, disassembly tasks, repair tasks, cleaning tasks and / or further inspection tasks of one or more of the plurality of parts.

[0465] The method according to any of the foregoing clauses further includes: determining the authorization status of the subsequent task based on the conditions of the plurality of parts and customer-specified requirements.

[0466] According to any of the foregoing provisions of the method, the authorization status of the subsequent task is further determined based on lifetime and durability data associated with the component.

[0467] The method according to any of the foregoing clauses, wherein the image capture system includes an image sensor system and a positioning system; and further includes: identifying multiple locations for image capture during the inspection work range based on an inspection recipe; wherein the capture instructions include: machine instructions of the positioning system for positioning one or more sensors of the image sensor system relative to the component; and machine instructions of the one or more sensors of the image sensor system for capturing an image.

[0468] According to any of the foregoing clauses, the positioning system includes a gantry configured to move one or more sensors of the image sensor system around the component.

[0469] According to any of the foregoing clauses, the positioning system includes a plurality of gantry frames, each gantry frame being coupled to one or more sensors of the image sensor system to form an imaging gantry.

[0470] According to any of the foregoing clauses, one or more sensors of the image sensor system are mounted on a gantry, the gantry is configured to move the one or more sensors in a first plane, and a track system is configured to move the gantry in a direction perpendicular to the first plane.

[0471] The method according to any of the foregoing clauses, wherein the positioning system includes a powered vehicle carrying one or more sensors of the image sensor system.

[0472] According to any of the foregoing provisions of the method, wherein the positioning system further includes a track defining the path of the powered vehicle.

[0473] The method according to any of the foregoing clauses, wherein the powered vehicle includes a drone.

[0474] The method according to any of the foregoing clauses, wherein the positioning system includes a cable-suspended camera positioning system.

[0475] The method according to any of the foregoing clauses, wherein the positioning system includes a programmable robotic arm.

[0476] According to any of the foregoing clauses, the positioning system is mounted on a bracket supporting the component.

[0477] The method according to any of the foregoing clauses, wherein the positioning system comprises two or more of a gantry system, track system, powered vehicle, component turntable or cable-suspended camera positioning system configured to simultaneously move different sensors of the image sensor system to perform the inspection.

[0478] The method according to any of the foregoing clauses further includes: moving the component relative to one or more stationary sensors of the image sensor system.

[0479] According to any of the foregoing clauses, the positioning system includes a turntable configured to rotate the component on an axis.

[0480] The method according to any of the foregoing clauses, wherein the image sensor system comprises one or more of an optical sensor, a LiDAR, a 3D scanner, an infrared sensor, a terahertz spectrometer, a microwave imaging sensor, an X-ray imager, a computed tomography scanner, an eddy current imaging sensor, or an ultrasound imager.

[0481] The method according to any of the foregoing clauses further includes: providing illumination instructions based on the directional lighting system.

[0482] According to any of the foregoing clauses, the inspection recipe specifies an indication image capture location associated with at least one desired image, wherein the indication capture location includes a coordinate position relative to the inspection space, the component, or a part of the component.

[0483] According to any of the foregoing clauses of the method, wherein the inspection formula specifies an indication of image capture orientation, including the roll, pitch, and / or yaw of the sensors of the image capture system.

[0484] The method according to any of the foregoing clauses further includes: determining the position and orientation of the component, wherein the plurality of positions are further identified based on the position and orientation of the component.

[0485] According to any of the foregoing provisions, the position and orientation of the component are determined by markings on a support that supports the component.

[0486] According to any of the foregoing clauses, the position and orientation of the component are determined based on markings on or near the component, wherein the markings include optical markings, color-coded markings, shape-coded markings, pattern-coded markings, embossed markings, engraved markings, sonar-readable markings, and / or lidar-readable markings.

[0487] According to any of the foregoing clauses, the plurality of locations includes sensor locations for locating one or more sensors of the image sensor system within the imaging space.

[0488] The method according to any of the foregoing clauses further includes: selecting at least one sensor of the image sensor system for each of the plurality of locations based on the inspection formula.

[0489] The method according to any of the foregoing clauses further includes: determining an image capture configuration for each of the plurality of locations based on the inspection recipe, and providing a capture configuration instruction to the image sensor system based on the image capture configuration.

[0490] The method according to any of the foregoing clauses, wherein the image capture configuration includes zoom level, focal length, aperture, exposure time, light sensitivity setting, illumination setting, image resolution, video length and / or sensor type selection.

[0491] The method according to any of the foregoing clauses further includes: determining an initial movement mode of the positioning system based on the plurality of locations.

[0492] The method according to any of the foregoing clauses further includes: selecting an additional capture location based on an image captured by the image sensor system, and providing updated machine instructions to the positioning system based on the additional capture location.

[0493] The method according to any of the foregoing clauses further includes: verifying the captured image based on the image requirements in the inspection recipe.

[0494] According to any of the foregoing clauses, the image requirements include focus, focus of the point of interest, blur amount, exposure, brightness, overlap with adjacent images, identification of parts of the component in the overlapping portion, and the presence of parts of the component.

[0495] According to any of the foregoing clauses, if the captured image does not meet the image requirements, the processor is configured to select a revised capture location and / or a revised capture configuration, and to provide machine instructions to the positioning system and / or the image sensor system based on the revised capture location and / or the revised capture configuration.

[0496] The method according to any of the foregoing clauses further includes: determining an image quality metric for the captured image; determining an adjusted capture configuration based on the image quality metric; and instructing the adjusted capture configuration to the image capture system to recapture the image.

[0497] According to any of the foregoing clauses, the adjusted capture configuration is determined based on a machine learning model trained with an image quality training dataset, which includes multiple captured images, each labeled with a capture location, capture configuration, and quality metric.

[0498] According to any of the foregoing provisions of the method, the machine instructions of the positioning system are determined based on simulating the movement of the positioning system using a computer model of the components.

[0499] The method according to any of the foregoing clauses further includes: determining the capture order of the plurality of locations, and wherein the machine instructions of the positioning system are determined based on the capture order.

[0500] The method according to any of the foregoing clauses further includes: determining an initial operating range of the image capture system based on an inspection recipe; communicating instructions to the image capture system based on the initial operating range; recognizing trigger conditions based on the captured images; determining an adaptation task based on the trigger conditions; and communicating updated instructions to the image capture system based on the adaptation task during the initial operating range.

[0501] According to any of the foregoing clauses, the inspection recipe includes image data requirements; and the triggering condition is identified based on comparing the captured image with the image data requirements of the inspection recipe.

[0502] According to any of the foregoing clauses, the method wherein the inspection formula identifies the part of the component and verifies the captured image by comparing the captured image with a computer model of the part, a previously captured image from the part, and / or a previously captured image from a similar part.

[0503] The method according to any of the foregoing clauses, wherein the inspection recipe is generated based on a machine learning model trained on multiple image sets captured from multiple engines.

[0504] According to any of the foregoing clauses, the inspection recipe is generated based on mapping a reference image from a maintenance manual associated with the component to a computer model of the component using a computer vision algorithm.

[0505] According to the method described in any of the foregoing clauses, the inspection formula and / or the adaptation task are determined based on the engine identifier, engine inspection history, engine construction record, engine repair history, customer-specified task requirements and / or identified problems.

[0506] According to any of the foregoing clauses, the inspection recipe is determined based on a recipe machine learning model trained on multiple images, the multiple images including labeled image quality metrics, component identifiers, component repair history, geographic regions associated with the component, component flight path history, and / or component operator identifiers.

[0507] According to any of the foregoing clauses of the method, the plurality of images include images of other components of the same or similar type, images of component models, and computer simulations of said components.

[0508] The method according to any of the foregoing clauses further includes: verifying the image quality of the captured image, wherein the triggering condition includes the captured image failing verification.

[0509] According to any of the foregoing provisions, the image quality includes image resolution, illumination, sharpness, and / or framing of the region of interest.

[0510] According to any of the foregoing clauses, the captured image is verified based on whether a specified part of the component identified in the inspection recipe is visible in the captured image, based on reference image comparison, machine object detection, machine feature defects, object recognition algorithm and / or optical character recognition algorithm.

[0511] According to any of the foregoing provisions of the method, the triggering condition includes the detection of anomalies and / or regions of interest in the captured image.

[0512] According to any of the foregoing provisions, the triggering condition is detected based on conditional evaluation of the component using a machine learning model trained on the sample inspection image and conditions associated with the sample inspection image.

[0513] According to any of the foregoing provisions, the triggering conditions are further detected based on the component identifier, component repair history, geographic region associated with component use, component flight path history, and / or component operator identifier.

[0514] According to any of the foregoing provisions, the triggering condition is detected based on multiple captured images.

[0515] According to any of the foregoing provisions, the triggering conditions are detected using a conditional machine learning model trained on multiple images labeled with the triggering conditions.

[0516] According to any of the foregoing clauses, the adaptation task includes an image capture task at a new capture location and / or with a new capture configuration.

[0517] According to any of the foregoing clauses, the new capture configuration includes zoom level, exposure time, light sensitivity setting, illumination setting, image resolution, video length, and / or sensor type selection.

[0518] According to any of the foregoing provisions of the method, the new capture location is determined based on the estimated location of a region of interest determined based on one or more captured images.

[0519] According to any of the foregoing provisions, the new capture position is determined based on simulating the movement of the positioning system of the image capture system using a computer model of the component.

[0520] According to any of the foregoing clauses, the instructions for moving the positioning system are simulated using a machine learning model trained on real-world and / or simulated positioning system movements.

[0521] According to any of the preceding clauses of the method, the new capture location and / or the new capture configuration is determined based on an image capture machine learning model trained on a training dataset, the training dataset comprising multiple images labeled with image quality metrics, capture locations, and capture configurations.

[0522] According to any of the foregoing clauses, the new capture configuration is determined based on image quality analysis performed using one or more captured images.

[0523] According to any of the foregoing provisions, the adaptation task includes a detailed query task for the identified region of interest.

[0524] The method according to any of the foregoing clauses, wherein the adaptation task includes the task of capturing multiple images with different capture configurations.

[0525] According to any of the foregoing clauses, the adaptation task includes the task of capturing multiple images at multiple additional capture locations and / or at multiple additional capture angles.

[0526] According to any of the foregoing clauses, the adaptation task includes a set of predefined capture locations around the identified region of interest.

[0527] The method according to any of the foregoing clauses, wherein the adaptation task includes a repair or maintenance task.

[0528] According to any of the foregoing clauses of the method, the captured image is captured by a first sensor of the image capture system, and the adaptive capture command is configured to cause a second sensor of the image capture system to capture the image.

[0529] According to any of the foregoing clauses, the captured image is captured using a first capture configuration, and the adaptive capture instruction is configured to cause the image capture system to capture the image using a second capture configuration.

[0530] According to any of the foregoing provisions, the updated instructions are determined based on a localization system motion machine learning model, which is configured to determine a path from a first capture location of the captured image to a second capture location of the adaptation task.

[0531] The method according to any of the foregoing clauses further includes: providing a reviewer user interface on a user interface device; transmitting the captured image and / or the adaptation task to the reviewer user interface for display; and determining instructions for the adaptation task based on user input received via the reviewer user interface.

[0532] The method according to any of the foregoing clauses further includes: providing an inspection user interface on a user interface device; receiving user input for an inspection task via the inspection user interface; and determining and communicating machine instructions for one or more sensors of the positioning system and the sensor capture system based on the user input.

[0533] According to any of the foregoing clauses, the inspection user interface includes controls for a plurality of sensors of the sensor system and a plurality of positioning devices of the positioning system, the controls being selectively and simultaneously displayed on the display of the user interface device.

[0534] According to any of the foregoing clauses, the inspection user interface includes a representation of the components of the aircraft and a representation of the sensors of the sensor system, the representation of the sensors being draggable and positionable around the representation of the components.

[0535] According to any of the foregoing clauses, the positioning system includes an engine rotation device configured to rotate parts within the engine.

[0536] The method according to any of the foregoing clauses, wherein the parts include turbine blades, disks, impellers, and / or shafts.

[0537] According to any of the foregoing provisions of the method, the inspection user interface includes a first slider for controlling the rotation of the engine rotation device.

[0538] According to any of the foregoing clauses, the inspection user interface further includes an image of the engine for selecting the insertion position of the endoscope.

[0539] The method according to any of the foregoing clauses further includes: determining the acquisition timing of the sensor system based on the movement of the positioning system, and providing the machine instructions to the sensor system based on the acquisition timing.

[0540] According to any of the foregoing clauses, the positioning system includes a component positioning system and a sensor positioning system, and the processor is configured to simultaneously control the movement of the component positioning system and the sensor positioning system to affect the relative positions of the sensor system and the component used for image capture.

[0541] The method according to any of the foregoing clauses, wherein the user interface device includes a head-mounted display.

[0542] According to any of the foregoing clauses of the method, wherein the user input includes movement and orientation of the head-mounted display, and the processor is configured to control movement of the image capture device via the positioning system based on the movement and orientation of the head-mounted display.

[0543] The method according to any of the foregoing clauses, wherein the user input includes touch input received on a touch-sensitive display showing an image of the component; and further includes: determining one or more capture positions based on the touch input; determining movement of one or more sensors of the image capture system based on the one or more capture positions; and communicating machine instructions to the image capture system to cause one or more sensors of the image capture system to move to the one or more capture positions.

[0544] According to any of the foregoing descriptions, the touch input includes tapping, dragging, and / or multi-touch squeezing or stretching actions.

[0545] According to any of the foregoing clauses, the dragging action corresponds to movement in a plane parallel to the image of the component, and the squeezing or stretching action corresponds to forward or backward movement relative to the plane.

[0546] The method according to any of the foregoing clauses further includes: displaying a previously captured image or model of the component; receiving a user selection of a region of interest in the previously captured image as the user input; and determining a capture location and / or capture configuration based on the region of interest.

[0547] According to any of the foregoing provisions, the capture location and the capture configuration are selected to focus and magnify the region of interest.

[0548] The method according to any of the foregoing clauses, wherein the inspection user interface includes a virtual, augmented, or mixed reality display.

[0549] According to any of the foregoing clauses, the controls of the image capture system are displayed as an overlay on the view of the sensor system.

[0550] According to any of the foregoing provisions of the method, the user input includes motion input captured by a motion sensor.

[0551] According to any of the foregoing clauses, the user interface device includes a control column or control lever for controlling the movement of the positioning system and / or for reviewing images captured by the image capture system.

[0552] The method according to any of the foregoing clauses further includes: receiving a captured image with additional metadata from the image capture system; and selectively displaying a subset of available controls of the positioning system and the sensor system based on the metadata of the captured image.

[0553] According to any of the foregoing clauses, the subset of available controls is selected based on the location and / or part identifier indicated in the metadata of the captured image.

[0554] The method according to any of the foregoing clauses further includes: retrieving a captured image of a component captured by the image capture system and metadata associated with the captured image; retrieving supplementary information from a database based on the metadata associated with the image; providing an inspection user interface for display on a user interface device; and displaying the image together with the supplementary information in the inspection user interface.

[0555] According to the method described in any of the foregoing clauses, the metadata identifies a view or part of the component, and the supplementary information includes technical or maintenance manual data associated with the part of the component.

[0556] According to the method described in any of the foregoing clauses, the metadata identifies a problem with a part of the component, and the supplementary information includes repair options associated with the problem.

[0557] The method according to any of the foregoing clauses further includes: identifying a role associated with a user of the user interface device; retrieving a report template from a report template database based on the role associated with the user; and selectively populating the report template using captured images and associated metadata stored in an inspection database.

[0558] According to any of the foregoing clauses, the method wherein the report template is generated by a report machine learning model trained on historical reports from multiple roles.

[0559] The method described according to any of the foregoing clauses, wherein the role is selected from the roles of inspector, administrator, customer, seller, and reviewer.

[0560] This written description uses examples to disclose this disclosure, including best practices, and also enables any person skilled in the art to practice this disclosure, including making and using any device or system and methods of making any combination. The patentable scope of this disclosure is defined by the claims, but may include other examples that would occur to a person skilled in the art. Such other examples are intended to fall within the scope of the claims if they include structural elements that are not indistinguishable from the literal language of the claims, or if they include equivalent structural elements that are not substantially different from the literal language of the claims.

Claims

1. An aircraft component inspection system, characterized in that, include: An image capture system, the image capture system comprising: Image sensor system; Positioning system; and A processor, communicatively connected to the image sensor system and the positioning system, is configured to: The inspection formula is determined at least based on identifiers associated with the components of the aircraft being inspected; Based on the inspection recipe, multiple locations are identified during image capture in the inspection workflow; Provide machine instructions to the positioning system to position the image sensor system relative to the components based on the plurality of locations; The image sensor system captures images at the multiple locations; and The image with captured location data is stored in the inspection data database.

2. The aircraft component inspection system according to claim 1, characterized in that, in, The inspection recipe specifies an indication image capture location associated with at least one desired image, wherein the indication image capture location includes a coordinate position relative to the inspection space, the component, or a part of the component.

3. The aircraft component inspection system according to claim 1, characterized in that, in, The inspection formula specifies an indication of image capture orientation, including the roll, pitch, and / or yaw of the sensors in the image sensor system.

4. The aircraft component inspection system according to claim 1, characterized in that, in, The inspection recipe specifies a view of the component for at least one desired image, wherein the view of the component defines the size and / or orientation of a portion of the component within an image frame.

5. The aircraft component inspection system according to claim 1, characterized in that, in, The processor is further configured to provide capture instructions to the image capture system based on the location of the image capture system or a portion thereof.

6. The aircraft component inspection system according to claim 1, characterized in that, in, The positioning system includes a snake-arm robot, and the machine commands include instructions for controlling the movement of the snake-arm robot.

7. The aircraft component inspection system according to claim 1, characterized in that, in, The processor is configured to verify the captured image before storing it in the inspection data database.

8. The aircraft component inspection system according to claim 7, characterized in that, in, The captured images are verified based on the location and / or image quality of the image capture system.

9. The aircraft component inspection system according to claim 1, characterized in that, in, The processor is further configured to determine the initial movement mode of the positioning system based on the plurality of locations.

10. The aircraft component inspection system according to claim 1, characterized in that, in, The processor is configured to determine an image quality metric for the image; The adjusted capture configuration is determined based on the image quality metric; as well as The adjusted capture configuration is instructed to the image capture system to recapture at least one of the images.

Images