Abnormality detection system for mobile wire harness

Abstract

The invention relates to a mobile wire harness anomaly detection system. An aerospace harness inspection system (211) includes a mobile personal computing device (240); a camera system (213) connected to the mobile personal computing device (240), a machine learning model (132, 224, 400, 500, 804, 1005, 1300) running in the mobile personal computing device (240); and a controller. A machine learning model (132, 224, 400, 500, 804, 1005, 1300) is trained to detect anomalies (205) in a set of images (129, 220) of a wire harness in an aerospace vehicle (206). The controller is configured to control the camera system (213) to generate a set of images (129, 220) of the wire harness; sending the set of images (129, 220) of the wire harness to a machine learning model (132, 224, 400, 500, 804, 1005, 1300); and receive a result (226) from the machine learning model (132, 224, 400, 500, 804, 1005, 1300) indicating whether there is an anomaly in the wire harness.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 516,215, filed on July 28, 2023, and entitled “Mobile Wiring Harness Anomaly Detection System,” which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to inspection, and more particularly to inspection of a vehicle's wiring system during manufacture and service. Background Art

[0004] Wiring harnesses are installed in different vehicles such as aircraft, ground vehicles, ships, spacecraft, and other types of vehicles. A wiring harness is an assembly of cables or wires that transmit electrical signals or power. These cables are secured or bundled together by components such as bands, cable ties, sleeves, tapes, conduits, or other types of components.

[0005] Wiring harnesses are used in vehicles to help simplify the installation of different electrical components. Using wiring harnesses can also reduce time and installation costs while improving the feasibility of system rubber. For example, aircraft wiring harnesses can be used to connect different electrical components (such as avionics equipment, navigation systems, computers, lighting systems, entertainment systems, and other systems and components in an aircraft).

[0006] The use of wire harnesses and aircraft enables many wires to be bundled into a single unit. The wires in the wire harness can have a specific and known path, which in time reduces the complexity of installation and inspection. In addition, the wire harness is designed to withstand various environmental conditions and can increase the safety and reliability of electronic systems in the aircraft. Summary of the invention

[0007] Embodiments of the present disclosure provide an aerospace wire harness inspection system, which includes a mobile personal computing device; a camera system connected to the mobile personal computing device; a machine learning model running in the mobile personal computing device; and a controller. The machine learning model is trained to detect anomalies in a set of images of a wire harness in an aerospace vehicle. The controller is configured to control the camera system to generate a set of images of the wire harness; send the set of images of the wire harness to the machine learning model; and receive a result from the machine learning model indicating whether there is an anomaly in the wire harness.

[0008] Another embodiment of the present disclosure provides a platform wiring inspection system, which includes a mobile personal computing device; a camera system connected to the mobile personal computing device; a machine learning model running in the mobile personal computing device; and a controller. The machine learning model is trained to detect anomalies in images of a wiring harness for a platform. The controller is configured to control the camera system to generate a set of images of the wiring harness; send the set of images of the wiring harness to the machine learning model; and receive a result from the machine learning model indicating whether there is an anomaly in the wiring harness.

[0009] Another embodiment of the present disclosure provides a method for inspecting a wire harness in an aerospace vehicle. A computer system controls a camera system to generate a set of images of a wire harness in the vehicle, wherein the camera system is connected to a mobile personal computing device. The computer system sends the set of images of the wire harness to a machine learning model, wherein the machine learning model is trained to detect anomalies in the set of images of the wire harness in the aerospace vehicle. The computer system receives a result from the machine learning model indicating whether an anomaly exists in the wire harness in the aerospace vehicle.

[0010] These features and functions can be achieved independently in various embodiments of the present disclosure or may be combined in yet other embodiments, in which further details can be seen with reference to the following description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The features believed to be novel characteristics of the illustrative embodiments are set forth in the appended claims. However, the illustrative embodiments together with the preferred modes of use, further objects and features thereof will be best understood by reference to the following detailed description of illustrative embodiments of the present disclosure when read in conjunction with the accompanying drawings, in which:

[0012] Figure 1 is a pictorial representation of a network in which an illustrative embodiment data processing system may be implemented;

[0013] Figure 2 is an illustration of a block diagram of an inspection environment according to an illustrative embodiment;

[0014] Figure 3 is an illustration of object recognition using compilation of neural network based computer vision outputs in accordance with an illustrative embodiment;

[0015] Figure 4 is an illustration of anomaly detection using a machine learning model in accordance with an illustrative embodiment;

[0016] Figure 5 is a diagram of machine learning model training according to an illustrative embodiment;

[0017] Figure 6is an illustration of wires and cables with anomalies according to an illustrative embodiment;

[0018] Figure 7 is a diagram of a table of wiring components and anomalies and recommended dispositions for the wiring components according to an illustrative embodiment;

[0019] Figure 8 is an illustration of a data flow graph for training a machine learning model using a synthetic dataset in accordance with an illustrative embodiment;

[0020] Fig. 9 is an illustration of using information from an engineering standards manual to create a computer-aided design model and a data flow diagram of actions for different anomalies in accordance with an illustrative embodiment;

[0021] Fig.10 is an illustration of a data flow graph for performing actions to rework a wiring harness according to an illustrative embodiment;

[0022] Fig.11 is an illustration of a data flow graph for transfer learning of a machine learning model used in a handheld scanner in accordance with an illustrative embodiment;

[0023] Fig.12 is an illustration of teacher-student transfer learning according to an illustrative embodiment;

[0024] Fig.13 is a diagram of a feedback loop for machine learning model performance enhancement according to an illustrative embodiment;

[0025] Fig.14 is an illustration of a data flow diagram for image augmentation according to an illustrative embodiment;

[0026] Fig.15 is an illustration of a data flow graph for labeling and parameter extraction according to an illustrative embodiment;

[0027] Fig.16 is an illustration of a data flow graph for composite image generation according to an illustrative embodiment;

[0028] Fig.17 is an illustration of a flow chart of a process for inspecting a wiring harness in an aerospace vehicle in accordance with an illustrative embodiment;

[0029] Fig.18 is an illustration of a flowchart of a process for creating training data according to an illustrative embodiment;

[0030] Fig.19 is an illustration of a flowchart of a process for selecting an image according to an illustrative embodiment;

[0031] Fig. 20is an illustration of a flow chart of a process for generating training data according to an illustrative embodiment;

[0032] Fig.21 is an illustration of a flow chart of a process for generating training data according to an illustrative embodiment;

[0033] Fig. 22 is an illustration of a flowchart of a process for training a machine learning model for use in a mobile personal computing device in accordance with an illustrative embodiment;

[0034] Fig.23 is an illustration of a flow chart of a process for performing remedial action according to an illustrative embodiment;

[0035] Fig.24 is an illustration of a block diagram of a data processing system according to an illustrative embodiment;

[0036] Fig.25 is an illustration of an aircraft manufacturing and service method according to an illustrative embodiment; and

[0037] Fig.26 is an illustration of a block diagram of an aircraft in which the illustrative embodiments may be implemented. DETAILED DESCRIPTION

[0038] The illustrative embodiments recognize and take into account one or more different considerations. For example, monitoring the health of the wiring side system is important in determining the appropriate maintenance and rework required. In the case of a large commercial aircraft, the amount of wiring may exceed 100 miles. For example, a commercial aircraft may have 330 miles or more of wiring. In addition, through the use of wiring harnesses, a complex network of interconnecting cables extends through the aircraft connecting various system components.

[0039] Therefore, inspecting the different wiring harnesses in an aircraft may be more difficult than desired.

[0040] Thus, an illustrative example provides a system, apparatus, method, and computer program product for inspecting a wiring harness of a platform, such as an aerospace vehicle. An aerospace wiring harness inspection system includes a mobile personal computing device; a camera system connected to the mobile personal computing device; a machine learning model running in the mobile personal computing device; and a controller. The machine learning model is trained to detect anomalies in a set of images of a wiring harness in an aerospace vehicle. The controller is configured to control the camera system to generate a set of images of the wiring harness; send the set of images of the wiring harness to the machine learning model; and receive a result from the machine learning model indicating whether an anomaly exists in the wiring harness.

[0041] In addition, the illustrative examples provide the ability to generate a sufficient amount of training data to train a machine learning model to detect anomalies in a wiring harness with a desired level of accuracy. Additional training data can be added to the training data sets of different examples using image augmentation, image creation using a model, and anomaly definitions from an engineering standards manual that defines and describes anomalies for wire and cable systems. In addition, the illustrative examples provide the ability to increase the accuracy in detecting anomalies for a particular fleet over time through a feedback loop. These and other illustrative features are described herein.

[0042] Reference is now made to the drawings, and in particular to the Figure 1 , depicts a pictorial representation of a network of data processing systems in which the illustrative embodiments may be implemented. Network data processing system 100 is a network of computers in which the illustrative embodiments may be implemented. Network data processing system 100 contains network 102, which is the medium used to provide communications links between various devices and computers connected together within network data processing system 100. Network 102 may include connections such as wires, wireless communication links, or fiber optic cables.

[0043] In the depicted example, server computer 104 and server computer 106 are connected to network 102 along with storage unit 108. In addition, client device 110 is connected to network 102. Client device 110 can be, for example, a computer, a workstation, a network computer, a vehicle, a machine, an appliance, or other device that can process data. As depicted in this example, client device 110 includes client computer 112, extension pole 114, handheld scanner 116, mobile phone 118, tablet computer 120, and smart glasses 122. Client device 110 can also include, for example, a computer, a workstation, or a network computer. In the depicted example, server computer 104 provides information to client device 110, such as boot files, operating system images, and applications.

[0044] In the depicted example, server computer 104 provides information, such as boot files, operating system images, and applications to client device 110. Furthermore, in this illustrative example, server computer 104, server computer 106, storage unit 108, and client device 110 are network devices connected to network 102, where network 102 is a communication medium for these network devices. Some or all of client devices 110 may form an Internet of Things (IoT), where these physical devices may be connected to network 102 and exchange information with each other through network 102.

[0045] In this example, client device 110 is a client to server computer 104. Network data processing system 100 may include additional server computers, client computers, and other devices not shown. Client device 110 connects to network 102 using at least one of a wired, fiber optic, or wireless connection.

[0046] Program instructions located in network data processing system 100 may be stored on a computer recordable storage medium and downloaded to the data processing system or other devices for use. For example, program instructions may be stored on a computer recordable storage medium on server computer 104 and downloaded to client device 110 via network 102 for use on client device 110.

[0047] In the depicted example, network data processing system 100 is the Internet with network 102 representing a global collection of networks and gateways that use the Transmission Control Protocol / Internet Protocol (TCP / IP) protocol suite or other network protocols to communicate with each other. At the heart of the Internet is a backbone of high-speed data communication lines between master nodes or host computers consisting of thousands of commercial, government, educational, and other computer systems that route data and messages. Of course, network data processing system 100 may also be implemented using several different types of networks. For example, network 102 may include at least one of the Internet, an intranet, a local area network (LAN), a metropolitan area network (MAN), or a wide area network (WAN). Figure 1 It is intended as an example and not as an architectural limitation for the different illustrative embodiments.

[0048] In this illustrative example, client device 110 includes personal mobile inspection device 111. Personal mobile inspection device 111 is any computing device that can be held or worn by a person. In addition, personal mobile inspection device 111 also includes a camera system that can generate image 129. In this example, personal mobile inspection device 111 includes extension pole 114, handheld scanner 116, mobile phone 118, tablet computer 120, and smart glasses 122, which can be used to inspect wiring harnesses in a platform such as commercial aircraft 130.

[0049] In one illustrative example, one or more of personal mobile inspection devices 111 may be operated by a human operator to generate images 129 of wiring harnesses in commercial aircraft 130 .

[0050] In one illustrative example, images 129 from these personal mobile inspection devices may be sent to inspection manager 131 via network 102. In this example, inspection manager 131 is located on server computer 104. As depicted, inspection manager 131 uses machine learning model 132 to determine if an anomaly is present in one or more of the wiring harnesses in commercial aircraft 130.

[0051] In another illustrative example, the personal mobile inspection device can operate in a standalone manner. For example, the handheld scanner 116 can generate and analyze images without sending the images to the inspection manager 131 for analysis. For this example, the inspection manager 133 and the machine learning model 134 are located in the handheld scanner 116. With this embodiment, the machine learning model 134 has a smaller size / scale than the machine learning model 132. The size / scale of the machine learning model 134 is selected so that the machine learning model 134 can be located on the handheld scanner 116 and run on the handheld scanner 116. In this example, the machine learning model 134 can be created using knowledge distillation, in which the teacher machine learning model trains the student machine learning model. In this example, the machine learning model 134 is the resulting student model.

[0052] As used herein, "a number" when used with respect to an item refers to one or more items. For example, "a number of different types of networks" is one or more different types of networks.

[0053] Additionally, the phrase "at least one of" when used with a list of items means that different combinations of one or more of the listed items can be used, and only one of each item in the list may be required. In other words, "at least one of" means that any combination of items and numbers of items in the list can be used, but not all items in the list are required. An item can be a specific object, thing, or category.

[0054] For example, but not limited to, "at least one of item A, item B, or item C" may include: item A; item A and item B; or item B. This example may also include: item A, item B, and item C; or item B and item C. Of course, any combination of these items may exist. In some illustrative examples, "at least one of" may be, for example, but not limited to, two of item A; one of item B; and ten of item C; four of item B and seven of item C; or other suitable combinations.

[0055] Reference now Figure 2, depicts an illustration of a block diagram of an inspection environment according to an illustrative embodiment. In this illustrative example, inspection environment 200 includes a plurality of hardware (e.g., Figure 1 Components implemented in the hardware shown in the network data processing system 100 in FIG. In this example, the wire harness inspection system 202 can be operated to inspect the wire harness 203 in the platform 204. The inspection is performed to determine whether there is an anomaly 205 in the wire harness 203 in the platform 204.

[0056] In the depicted example, platform 204 takes the form of aerospace vehicle 206. Aerospace vehicle 206 may include both atmospheric vehicles and space vehicles. For example, aerospace vehicle 206 may be a commercial aircraft, a rotorcraft, a tiltrotor aircraft, a tiltrotor aircraft, a vertical take-off and landing aircraft, an electric vertical take-off and landing vehicle, a personal aerial vehicle, a spacecraft, a jet aircraft, and a space shuttle, or other types of aerospace vehicles. When used to inspect aerospace vehicle 206, wire harness inspection system 202 is aerospace wire harness inspection system 211.

[0057] In this illustrative example, wire harness inspection system 202 includes computer system 212 , camera system 213 , and inspection manager 214 . Camera system 213 includes one or more cameras. Camera system 213 is connected to computer system 212 .

[0058] In this example, the inspection manager 214 is located in the computer system 212. The inspection manager 214 can be implemented in software, hardware, firmware, or other commitments of a combination thereof. When software is used, the operations performed by the inspection manager 214 can be implemented in program instructions configured to run on hardware (such as a processor unit). When firmware is used, the operations performed by the inspection manager 214 can be implemented in program instructions and data and stored in persistent memory to run on the processor unit. When hardware is employed, the hardware may include circuits that operate to perform the operations in the inspection manager 214.

[0059] In an illustrative example, the hardware can take the form of at least one selected from a circuit system, an integrated circuit, an application specific integrated circuit (ASIC), a programmable logic device, or some other suitable type of hardware configured to perform several operations. Through a programmable logic device, the device can be configured to perform several operations. The device can be reconfigured at a later time, or can be permanently configured to perform several operations. Programmable logic devices include, for example, programmable logic arrays, programmable array logic, field programmable logic arrays, field programmable gate arrays, and other suitable hardware devices. Additionally, the process can be implemented in organic components integrated with inorganic components, and can be composed entirely of organic components that do not include humans. For example, the process can be implemented as a circuit in an organic semiconductor.

[0060] As used herein, "a number" when used with respect to an item refers to one or more items. For example, "a number of operations" is one or more operations.

[0061] Additionally, the phrase "at least one of" when used with a list of items means that different combinations of one or more of the listed items can be used, and only one of each item in the list may be required. In other words, "at least one of" means that any combination of items and numbers of items in the list can be used, but not all items in the list are required. An item can be a specific object, thing, or category.

[0062] For example, but not limited to, "at least one of item A, item B, or item C" may include item A; item A and item B; or item B. This example may also include item A, item B, and item C; or item B and item C. Of course, any combination of these items may exist. In some illustrative examples, "at least one of" may be, for example, but not limited to, two of item A; one of item B; and ten of item C; four of item B and seven of item C; or other suitable combinations.

[0063] Computer system 212 is a physical hardware system and includes one or more data processing systems. When there is more than one data processing system in computer system 212, these data processing systems communicate with each other using a communication medium. The communication medium may be a network. The data processing system may be selected from at least one of a computer, a server computer, a tablet computer, or some other suitable data processing system.

[0064] As depicted, computer system 212 includes several processor units 216 that are capable of executing program instructions 218 to implement the processes in the illustrative example. In other words, program instructions 218 are computer-readable program instructions.

[0065] As used herein, a processor unit in number of processor units 216 is a hardware device and includes hardware circuits, such as hardware circuits on an integrated circuit, that respond to and process instructions and program codes for operating a computer. When number of processor units 216 execute program instructions 218 for a process, number of processor units 216 may be one or more processor units in the same computer or in different computers. In other words, the process may be distributed among processor units 216 on the same computer or on different computers in computer system 212.

[0066] In addition, the number of processor units 216 can be the same type or different types of processor units. For example, the number of processor units 216 can be selected from at least one of a single core processor, a dual core processor, multiple processor cores, a general purpose central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or some other type of processor unit.

[0067] In this illustrative example, inspection manager 214 controls camera system 213 to generate a set of images 220 of wire bundle 222 in wire bundle 203. In this example, camera system 213 sends images 220 to inspection manager 214 for analysis. Inspection manager 214 sends set of images 220 to machine learning model 224. Machine learning model 224 may be implemented using a neural network, a random forest model, a decision tree, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative machine learning model trained using a generative adversarial network (GAN), a transfer learning model, and other suitable types of machine learning models.

[0068] In this example, machine learning model 224 is trained to detect anomaly 228 in set of images 220 of wire harness 222 in platform 204 , such as aerospace vehicle 206 . In response, inspection manager 214 receives result 226 from machine learning model indicating whether anomaly 228 is present in wire harness 222 .

[0069] In one illustrative example, computer system 212 includes mobile personal inspection device 240. For this example, mobile personal inspection device 240 may be a computing device that may be handheld or worn by human operator 241. Mobile personal inspection device 240 may be selected from the group consisting of an extension pole, a handheld scanner, a smartphone, a mobile phone, a tablet computer, smart glasses, augmented reality goggles, a laptop computer, a smart watch, or other devices.

[0070] In this example, the camera system 213 is connected to the mobile personal inspection device 240. When one component is "connected" to another component, the connection is a physical connection. For example, a first component (camera system 213) can be considered to be physically connected to a second component (mobile personal inspection device 240) by being fixed to the second component, bonded to the second component, mounted to the second component, welded to the second component, fastened to the second component, or connected to the second component in some other suitable manner. The first component can also be connected to the second component using a third component. The first component can also be considered to be physically connected to the second component by being formed as a part of the second component, an extension of the second component, or both.

[0071] For this example, mobile personal inspection device 240 generates image 220 using camera system 213 and sends image 220 to inspection manager 214 for analysis.

[0072] In another illustrative example, mobile personal inspection device 240 may operate as a standalone device without the need to send images over a network or wireless connection for analysis. For this example, inspection manager 214 and machine learning model 224 may be located, along with mobile personal inspection device 240.

[0073] In one illustrative example, there are one or more technical solutions that overcome the technical problem of identifying anomalies in a large length of a wiring harness in an aerospace vehicle. Thus, one or more solutions can provide a technical effect of enabling the creation of a machine learning model that provides a desired level of accuracy in identifying anomalies in a wiring harness located in a platform such as an aerospace vehicle.

[0074] Using the wire harness inspection system in various illustrative examples can result in more efficient identification of anomalies in wire harnesses compared to current techniques. Using a machine learning model with a specialized training data set and training operations for training the machine learning model that focuses on wire harnesses of aerospace vehicles can result in increased efficiency in identifying anomalies. In various illustrative examples, the training data set can be generated in a manner that provides an increased amount of training data for training the machine learning model to identify wire harness anomalies.

[0075] Furthermore, by using feedback loops and continuously trained machine learning models, aerospace harness inspection systems can improve accuracy in identifying anomalies. This increase in accuracy can be focused on specific fleets of aircraft by using the harness inspection system on these specific aircraft leads. Furthermore, by continuously training the machine learning model based on images from these fleets of aircraft, the machine learning model can more efficiently and accurately identify anomalies that occur over time that are specific to these fleets of aircraft.

[0076] Additionally, the aerospace harness inspection system can be implemented in mobile personal computing devices such as handheld scanners, mobile phones, smart glasses, and other devices. In these illustrative examples, machine learning models can be created for use in these mobile personal computing devices so that the mobile computing devices can be used as standalone devices without the need to send images to another location for analysis.

[0077] Computer system 212 may be configured to perform at least one of the steps, operations, or actions described in the different illustrative examples using software, hardware, firmware, or a combination thereof. Thus, computer system 212 operates as a special-purpose computer system, wherein inspection manager 214 in computer system 212 enables at least one of a more efficient or more accurate inspection to identify anomaly 205 and wiring harness 203. In particular, inspection manager 212 converts computer system 212 into a special-purpose computer system as compared to currently available general-purpose computer systems that do not have inspection manager 214.

[0078] Figure 2 The diagram of inspection environment 200 is not meant to imply physical or architectural limitations of the way in which the illustrative embodiments can be implemented. Other components other than or in place of the components shown can be used. Some components may be unnecessary. In addition, blocks are also presented to illustrate some functional components. When implemented in the illustrative embodiments, one or more of these blocks can be combined, divided or combined and divided into different blocks.

[0079] For example, the harness inspection system 202 can be used to inspect harnesses in other platforms in addition to or in place of the aerospace vehicle 206. For example, the platform 204 can be selected from the group consisting of a mobile platform, a fixed platform, a land-based structure, a water-based structure, a space-based structure, an aircraft, a commercial aircraft, a rotorcraft, a tilt-rotor aircraft, a tilt-rotor aircraft, a vertical take-off and landing aircraft, an electric vertical take-off and landing vehicle, a personal aerial vehicle, a surface vessel, a tank, a personnel transporter, a train, a spacecraft, a space station, a satellite, a submarine, an automobile, a power plant, a bridge, a dam, a house, a manufacturing facility, and a building. The harness inspection system 202 can also be referred to as an aerospace harness inspection system.

[0080] Next reference Figure 3 , depicts an illustration of object recognition using a compilation of neural network-based computer vision outputs according to an illustrative embodiment. In this example, a convolutional neural network (CNN) 300 is Figure 2 An example of an implementation of the machine learning model 224.

[0081] In this illustrative example, convolutional neural network 300 may receive camera image 302 containing wire harness 303 and generate bill of materials (BOM) 304 for anomalies identified for wire harness in camera image 302. In addition, bill of materials 304 may also include a history of anomalies, rework, repairs, or other actions taken with respect to wire harness 303.

[0082] In this illustrative example, convolutional neural network 300 uses a computer vision algorithm to perform object recognition 310. In this example, the computer vision algorithm for object recognition 310 may include several different components (such as image segmentation 312, object detection 314, image classification 316, object localization 318).

[0083] In this example, image segmentation 312 may include separating pixels in camera image 302 to create segments. These segments may separate line bundle 303 from the background. This separation facilitates identifying the line bundle in camera image 302.

[0084] Object detection 314 may include detecting specific objects in camera image 302. These objects may be specific objects such as line bundles 303 that convolutional neural network 300 has been trained to identify from image pixels in camera image 302.

[0085] In this example, image classification 316 can be used to determine whether an anomaly is present in camera image 302. In some cases, the classification can include no anomaly in addition to different types of anomalies. In addition, image classification 316 can also be used for image selection and labeling. The selection and labeling of images can be used to create a training data set that is used to train a machine learning model to classify the type of anomaly in the image.

[0086] Object localization 318 may include detecting bounding boxes in camera image 302 that contain specific objects, such as wire harness 303. In this example, convolutional neural network 300 has been trained to identify wire harnesses.

[0087] Next reference Figure 4, depicts an illustration of anomaly detection using a machine learning model according to an illustrative embodiment. As depicted in this example, anomaly detection can be performed using a machine learning (ML) model 400. In this example, the machine learning model 400 performs computer vision (CV) based object recognition 403.

[0088] In this example, machine learning model 400 receives image 402 as input for analysis. In this example, machine learning model 400 identifies anomaly 404 and outputs image 407 with a magnified view of image 402 showing anomaly 404. In this example, anomaly 404 is indicated using anomaly bounding box 406. In this example, machine learning model 400 performs this analysis and identification of anomaly 404 using computer vision-based object recognition, which may include, for example, Figure 3 Components of components of object identification 310 are shown.

[0089] Next, in Figure 5 , a diagram of machine learning model training is depicted according to an illustrative embodiment. In this illustrative example, the machine learning model 500 is implemented as a wiring anomaly classifier 502.

[0090] In this example, model training and validation 504 is performed on the machine learning model 500 to train the machine learning model 500 to operate as a wiring anomaly classifier 502 .

[0091] In this example, training is performed using a training data set 506. The training data set 506 includes augmented data 508 generated from images and synthetic data 510. These types of data are used in addition to actual images to increase the amount of training data available for training the machine learning model 500.

[0092] A different illustrative example provides the ability to generate additional images for training data set 506 in a manner that increases the accuracy of machine learning model 500 operating as wiring anomaly classifier 502. As depicted in this example, images in augmented data 508 and synthetic data 510 are created in a manner that provides an increased number of images showing anomalies in a manner that increases the accuracy of machine learning model 500.

[0093] In this illustrative example, detected abnormal images 512 may be identified and stored as abnormal images 514. In this example, image data augmentation 516 may be performed on abnormal images 514 to create augmented images 517. These augmented images are stored as augmented data 508. In this illustrative example, the augmentation of abnormal images 514 may include at least one of translation, rotation, scaling, or other types of manipulation of abnormal images 514. Thus, the number of images that can be used to train machine learning model 500 may be increased by performing image data augmentation 516 on abnormal images 514. By increasing the number of images in training data set 506, the performance of machine learning model 500 may be improved by increasing the size of training data set 506. In this example, augmented data 508 includes abnormal images 514 and images generated from augmented abnormal images 514.

[0094] Additionally, the size of the training data set 506 may be increased by using synthetic data 510. In this example, the synthetic data 510 includes images that are created and not based on actual images (such as the detected anomaly image 512).

[0095] For example, the synthetic data 510 can be created using anomalies in anomaly definitions 520 defined in an engineering standards manual and a computer-aided design (CAD) model 522. These two sources of information can be used to create an image that forms the synthetic data 510. In this example, the identification of anomalies as defined in the engineering standards manual and other sources can be used to add those anomalies in the CAD model 522, or to generate an image with anomalies using the CAD model 522. The CAD model 522 with anomalies based on the anomaly definitions 520 can then be used to generate a synthetic image 524 that forms the synthetic data 510. In this example, snapshot images of the CAD model 522 with anomalies can be acquired from different orientation perspectives and can be generated to create a synthetic image 524. In this example, the synthetic image 524 is a synthetic harness image 525 with anomalies 527. These images are synthetic wiring anomaly images.

[0096] In addition, anomaly definition 520 may also include an identifier of the anomaly type (such as a fold, wrinkle, scratch, slight wear, broken bond, conducted information, or other types of anomalies. These identifiers may be used as labels when generating composite image 524 using CAD model 522.

[0097] The illustration of training data set generation in this example is provided as an example implementation and is not meant to limit the manner in which other illustrative examples may be implemented. For example, image data augmentation 516 may be performed on synthetic image 524 to create additional images for augmented image 517 in addition to those images created using abnormal image 514. As another example, although not shown, image and augmented data 508 and synthetic data 510 may include labels depending on the type of training used to train machine learning model 500.

[0098] Thus, by using the anomaly types defined in the anomaly definitions 520 as labels, the machine learning model 500 can output classifications based on the anomaly types in the anomaly definitions 520 after performing model training and validation 504 using the training data set 506. In this way, the classifications generated by the machine learning model 500 can conform to desired definitions (such as those used by an industry, manufacturer, supplier, airline, or other organization).

[0099] Next reference Figure 6 and Figure 7 , depicts a graphical representation of anomaly definitions according to an illustrative embodiment. The two figures are Figure 5 Example of exception definition 520 in FIG.

[0100] In this illustrative example, Figure 6 is an illustration of wires and cables with anomalies depicted in accordance with an illustrative embodiment. In this illustrative example, illustration 600 is an example of anomalies that can be used with a computer-aided design model to generate a composite image. As depicted, illustration 600 details the types of anomalies that may occur on wires and cables. These details in illustration 600 illustrate the types of anomalies that can be used to generate an image of the anomaly using a computer-aided design model.

[0101] refer to Figure 7 , depicts a table of wire components and an illustration of anomalies for wire components with recommended treatments according to an illustrative embodiment. In this illustrative example, table 700 includes columns in the form of wire components 702, anomaly conditions 704, and recommended treatments.

[0102] In another illustrative example, the exception type may also be based on the recommended disposition 706. For example, the exception type may be replacement, use as is, insulation repair, or some other type of classification. In other illustrative examples, the exception type may be associated with the recommended disposition 706 to indicate repair, rework, or no action that may be performed on the wiring harness.

[0103] In this illustrative example, abnormal condition 704 can be used as a label to associate an illustration (such as Figure 6600) is used with the computer-aided design model to classify the composite image. In another illustrative example, the exception type can also be based on the recommended treatment 706. For example, the exception type can be replaced, used as is, insulation repair, or some other type of classification. In other illustrative examples, the exception type can be associated with the recommended treatment 706 to indicate an action that can be performed on the wiring harness.

[0104] Next go to Figure 8 , depicts an illustration of a data flow diagram for training a machine learning model using a synthetic dataset according to an illustrative embodiment. In this illustrative example, the synthetic dataset 800 is Figure 5 An example of synthetic data 510 with labels in .

[0105] As depicted, training dataset 810 can be used to perform machine learning (ML) model training 812 to train machine learning model 804 to classify anomalies. In this example, the amount of training data in training dataset 810 can be increased by generating synthetic dataset 800 and adding the images in synthetic dataset 800 to training dataset 810. In this example, synthetic dataset 800 can be generated using generative artificial intelligence (AI) model 802 and machine learning (ML) model 804.

[0106] In this illustrative example, bounding box image 814 is an image within a bounding box from training dataset 810. In these examples, bounding box image 814 is an image that contains anomalies.

[0107] In this illustrative example, the training data set 810 is Figure 5 8. Thus, the training dataset 810 may include images in the form of augmented data and synthetic data.

[0108] As depicted, generative AI model 802 generates composite wiring anomaly image 806 using bounding box image 814. Composite wiring anomaly image 806 is composite wire harness image 831 with anomaly 833 in this example.

[0109] Generative AI model 802 is a generative model from a generative adversarial network (GAN). Generative AI model 802 has been trained to generate a synthetic wiring anomaly image 806. Generative AI model 802 has been trained in the statistical distribution of the data and learns knowledge about the data to generate a synthetic wiring anomaly image 806.

[0110] In this illustrative example, before starting this data flow, the machine learning model 804 has been trained using the training data set 810. In this example, the machine learning model 804 receives as input the composite wiring anomaly images 806. The machine learning model 804 classifies these images to generate composite wiring anomaly labels 808. The classification involves detecting anomalies and classifying the anomalies in the composite wiring anomaly images 806.

[0111] In this example, join 820 associates the composite wiring anomaly labels with the composite wiring anomaly images 806 in terms of weights to create composite data set 800 .

[0112] In this example, synthetic data set 800 is added to training data set 810 to increase the amount of training data in training data set 810. By adding synthetic data set 800 to training data set 810, machine learning model training 812 can be performed for machine learning model 804. This further training with the added data in training data set 810 can increase the accuracy of machine learning model 804.

[0113] Multiple iterations of this process may be performed to increase the size of the training data set 810. In this example, the synthesized wiring anomaly labels and weights may be examined to determine the accuracy of the machine learning model 804. Through analysis, additional iterations of this process may be performed until satisfactory performance of the machine learning model 804 occurs.

[0114] Figure 8 The data flow in illustrates an application of using a generative artificial intelligence model, where the model is used to create additional synthetic data in the form of a synthetic wiring anomaly image 806. The generative artificial intelligence model 802 can be trained with an existing data set to create additional synthetic data (features only). In this example, the synthetic wiring anomaly label 808 is identified by a machine learning model 804 that has been trained for object detection. Therefore, the synthetic wiring anomaly image and label containing features in the form of anomalies can be concatenated together to create a synthetic data set 800, which is added to the training data set 810 to create a larger training data set to train a more accurate machine learning model.

[0115] Next reference Fig. 9, depicts an illustration of a data flow diagram using information from an engineering standards manual to create a computer-aided design model and actions for different anomalies according to an illustrative embodiment. In this illustrative example, the engineering standards manual 900 can be a manual for wiring and cable anomaly identification and remedial actions. In this illustrative example, information retrieval 902 can be performed to retrieve information about actions to be taken based on different wiring anomalies. This information can be used to create a wiring anomaly action table 904. The table can include information about actions to be taken relative to the wiring harness that identified the anomaly. In addition, anomaly selection and computer-aided design model creation 906 can be performed to create a computer-aided design (CAD) model 908 containing the anomaly. This creation of the CAD model 908 can be performed using the illustration and description of the anomaly in the engineering standards manual 900.

[0116] Next go to Fig.10 , an illustration of a data flow graph for performing actions to rework a wiring harness is depicted in accordance with an illustrative embodiment. In this illustrative example, an image 1000 includes an anomaly 1002 within a bounding box 1004. The image is input into a machine learning model 1005 operating as a wiring anomaly classifier 1006. In this example, the machine learning (ML) model 1005 operating as the wiring anomaly classifier 1006 is a Figure 5 An example of a machine learning model 500 for the operation of a wiring anomaly classifier 502 in FIG. Figure 5 The same training dataset as depicted was used for training.

[0117] In this illustrative example, the machine learning model 1005 outputs anomaly codes 1010. These anomaly codes correspond to various classifications for anomalies. The anomaly code 1010 can be text, a number, an alphanumeric string, or take other forms.

[0118] In this illustrative example, the exception code 1010 can be used as an index to the wiring exception action table 1012. The table is based on Fig. 9 900 . In this way, the exception code can be used to identify the action in the wiring exception action table 1012. Based on the identification, one or more remedial actions 1014 can be performed as rework of the wiring harness in which one or more anomalies have been identified. In some cases, the remedial action can be repair, rework, or no action. In this way, a machine learning model used with a mobile personal inspection device (such as a handheld scanner) can be used to identify anomalies in an actual physical wiring harness so that remedial actions can then be performed.

[0119] Next reference Fig.11, an illustration of a data flow graph for transfer learning of a machine learning model used in a handheld scanner is depicted according to an illustrative embodiment. In this illustrative example, a large machine learning (ML) model 1100 is located in a cloud 1102.

[0120] In this example, teacher-student transfer learning 1104 may be performed to create a small machine learning (ML) model 1106 for use in a handheld device 1108. This type of transfer learning creates a machine learning model with sufficient performance that can be run in the handheld device 1108 to perform real-time inspections that may include detecting anomalies and recommending remedial actions.

[0121] In this example, the large ML model 1100 is a machine learning model that has a size that cannot be run directly on the handheld device 1108. The small ML model 1106 has a footprint that is small enough to run on the handheld device 1108. The small machine learning model will have adequate performance in detecting and classifying anomalies in wiring parts.

[0122] In this way, the handheld device 1108 can be used for real-time vision-based harness health inspection in the field. In this example, the handheld device 1108 can be used as a standalone device without the need to send images back to the large machine learning model 1100 in the cloud 1102 for analysis.

[0123] In this way, the handheld device 1108 with one or more small machine learning models 1106 increases flexibility in performing wire harness inspections. In this example, the handheld device 1108 with the small machine learning model can perform real-time inspections, detections, and remedial action recommendations in the field under various conditions, including when a network or Internet connection is unavailable.

[0124] Go to Fig.12 , depicts an illustration of teacher-student transfer learning according to an illustrative embodiment. In this illustrative example, the teacher model 1200 is a pre-trained model. This pre-trained machine learning model has been trained to detect anomalies in a harness. The detection of anomalies may also include classifying the anomalies, and may also include recommending remedial actions to be taken based on the classification of the anomalies.

[0125] In this illustrative example, student model 1202 is a machine learning model to be trained by teacher model 1200. Student model 1202 has a smaller size than teacher model 1200. Student model 1202 is of a size selected to run on a mobile computing device such as handheld scanner 1204 and mobile phone 1206.

[0126] In this example, teacher-student transfer learning is also referred to as knowledge distillation 1208.

[0127] In this example, the teacher model 1200 processes the training data 1212 and generates predictions 1214. These predictions are considered soft labels using a softmax layer that converts the output from the teacher model 1200 into probabilities for the predictions 1214.

[0128] In this example, the student model 1202 processes the training data 1212 to generate predictions 1216. In this example, the student model 1202 learns to predict these softer probabilities in the predictions 1214, rather than the original hard labels for the training data 1212.

[0129] This process enables the student model 1202 to learn from the more expressive knowledge of the teacher model 1200. In addition, this type of training enables the student model 1202 to learn subtle patterns and relationships that the normal labels used in the training data 1212 may not convey.

[0130] In this example, the predictions 1214 become the ground truth 1218 that guides the student model 1202 during the training process. Even if the student model 1202 is smaller than the teacher model 1200, this type of training can provide more informative and efficient knowledge transfer and train the student model 1202 to have the desired performance.

[0131] By leveraging the predictions 1214 from the teacher model 1200, the student model 1202 can benefit from the learned representation and generalization capabilities of the teacher model 1200. This type of knowledge transfer can enable the student model 1202 to achieve comparable performance to the teacher model 1200, even though the student model 1202 is smaller in size. Thus, knowledge distillation 1208 creates an efficient machine learning model for deployment in resource-constrained environments, such as in handheld scanners 1204 and mobile phones 1206.

[0132] Next reference Fig.13 , depicts an illustration of a feedback loop for machine learning model performance enhancement according to an illustrative embodiment. In this illustrative example, a machine learning model 1300 has been trained by using a training data set 1304 in a regular loop 1305 for model training and validation 1302. In this example, the training data set 1304 includes augmented data 1306 and synthetic data 1308. In this example, the training data set 1304 with augmented data 1306 and synthetic data 1308 is Figure 5An example of a training dataset 506 with augmented data 508 and synthetic data 510 in FIG.

[0133] In this example, feedback loop 1307 can be used to perform additional training of machine learning model 1300. In this example, handheld scanner 1320 generates images. These images can be stored with images 1322 generated by handheld scanner 1320 and other handheld scanners or mobile personal scanning devices. In this example, image selection and tagging 1324 can be performed on images 1322. In this example, images 1322 can be targeted to a specific fleet of aircraft. In other words, images 1322 can be a fleet of aircraft selected from a group including: all aircraft operated by an organization, aircraft of the same type, aircraft on the same route, aircraft with the same cabin configuration, aircraft within a life span, or other aircraft groupings.

[0134] In this example, the images selected from images 1322 may be images that may provide additional model performance enhancements for machine learning model 1300. For example, the images selected from images 1322 may include images of anomalies that occurred for a particular fleet of aircraft for which scans were performed by handheld scanner 1320 and mobile personal computing devices. Over time, images 1322 may be for harnesses that have anomalies in different areas of the aircraft in the fleet. In other words, harnesses in some areas of the aircraft may have more anomalies over time than harnesses in other areas. Additionally, particular types of anomalies may occur more frequently over time, which may be specific to a fleet of aircraft.

[0135] By selecting these images, the machine learning model 1300 can be trained to better detect those areas in the aircraft fleet that are targeted using the handheld scanner 1320.

[0136] In this example, images selected from images 1322 are labeled by image selection and labeling 1324 to form labeled images 1326. In this example, labeled images 1326 can be processed by image data augmentation 1328 to generate augmented images for augmenting data 1306. In this way, new data can be added to training data set 1304 or used in training machine learning model 1300. In addition, labeled images 1326 can also be used by model training and validation 1302 together with training data set 1304 to train further machine learning models 1300.

[0137] The illustration of this feedback loop is an example of one way in which a machine learning model can be iteratively or continuously trained to improve performance. The illustration is not meant to limit the way in which other illustrative examples can be implemented. For example, in some illustrative examples, labeled image 1326 can be considered part of training data set 1304.

[0138] Reference now Fig.14 , an illustration of a data flow graph for image augmentation is depicted in accordance with an illustrative embodiment. In this illustrative example, image dataset 1400 may be processed in a number of different ways to generate image dataset 1400 and augmented image dataset 1402 .

[0139] For example, images from the image dataset 1400 may be selected using manual selection 1404 or automatic selection by training a machine learning (ML) model 1406. In this example, the selected image is an image with an identified abnormal region 1408, which may be augmented to create additional images for training data.

[0140] In this example, defect area resizing, rotation, and translation 1410 may be performed on the image selected for augmentation. At least one of these types of changes to the image or any combination of these changes may be used to create augmented images for augmented image dataset 1402. These images in augmented image dataset 1402 may be Figure 5 An example of amplification data 508 in FIG.

[0141] Next reference Fig.15 , depicts an illustration of a data flow graph for labeling and parameter extraction according to an illustrative embodiment. In this illustrative example, a wire harness image 1500 is an image captured by a camera. In this depicted example, manual annotation 1502 can be performed on the wire harness image to form a labeled training dataset 1504 for use in machine learning (ML) model training 1506.

[0142] Furthermore, in this example, image scene analysis 1508 may be performed on the wire harness image 1500 to obtain parameters 1510 for the wire harness image 1500. The parameters 1510 may be at least one of a geometry or a material of an object in the image. The analysis may be performed using an artificial intelligence model.

[0143] In this example, these parameters may be used for computer-aided design (CAD) model creation 1512 or synthetic image creation 1514 using a generative artificial intelligence model.

[0144] Next, in Fig.16, an illustration of a data flow graph for synthetic image generation is depicted in accordance with an illustrative embodiment. In this example, a generative artificial intelligence model 1600 is an example of an artificial intelligence model that can be used to generate parameters 1602 for use in creating an image of synthetic data.

[0145] In this example, parameters 1602 (such as the geometry and material of objects in the image) and details of the wiring defects (such as type, size, or orientation) are input into the generative AI model 1600, where the output parameters are added to the parameters 1602 to create a larger set of parameters 1602.

[0146] In this example, the generative AI model 1600 learns the statistical distribution of the parameters. The statistical distribution can be used to expand the training data set by creating a larger set of parameters with similar or identical statistical distributions. Therefore, the resulting larger training data set can help improve the accuracy of the trained machine learning model. In this example, the larger training data set takes the form of labeled synthetic images 1606.

[0147] In this example, parameters 1602 may be used as input to computer-aided design (CAD) software 1603 to create and / or modify a computer-aided design model 1604. In this example, snapshots 1608 may be created from the computer-aided design model 1604. In this example, the snapshots 1608 may be acquired from the computer-aided design model 1604 at different angles, at different distances, under different lighting conditions, or by simulating various camera settings. Since the image is created from a scene with known wiring anomalies, labels may be automatically added to the snapshots 1608 of the created image without the need for additional processing or manual data labeling to generate the labeled composite image 1606.

[0148] In this example, the labeled composite images 1606 may be images with or without wiring anomalies. These images may be labeled to indicate whether an anomaly exists. In addition, labels may be added for the type of anomaly. Thus, a generative artificial intelligence model may be used to increase the size of a training data set by creating parameters 1602 based on statistical distributions and by creating composite wiring anomaly images.

[0149] Next go to Fig.17 , an illustration of a flowchart of a process for inspecting wire harnesses in an aerospace vehicle is depicted in accordance with an illustrative embodiment. Fig.17 The processes in may be implemented in hardware, software, or both. When implemented in software, the processes may take the form of program instructions that are executed by one or more processor units in one or more hardware devices located in one or more computer systems. For example, the processes may be implemented in Figure 2The method is implemented in the inspection manager 214 in the computer system 212 in the embodiment.

[0150] The process begins by controlling a camera system to generate a set of images of a wiring harness in a vehicle, wherein the camera system is connected to a mobile personal computing device (operation 1700). The process sends the set of images of the wiring harness to a machine learning model, wherein the machine learning model is trained to detect anomalies in the set of images of the wiring harness in an aerospace vehicle (operation 1702).

[0151] The process receives results from the machine learning model indicating whether an anomaly is present in a wiring harness in the aerospace vehicle (operation 1704 ). The process then terminates.

[0152] Reference now Fig.18 , an illustration of a flowchart of a process for creating training data is depicted in accordance with an illustrative embodiment. The training data in this example can be used to train a machine learning model to be more specific in identifying anomalies in a particular fleet of aircraft. The process in this example is a flow chart that can be used to create training data in accordance with an illustrative embodiment. Fig.17 An example of additional operations performed by the operations in .

[0153] The process begins by storing images captured by a group of mobile personal computing devices, wherein the stored images captured from the group of mobile personal computing devices form stored images (operation 1800). The process stores, in association with the stored images, classifications of anomalies identified in the images by the group of mobile personal computing devices (operation 1802).

[0154] The process selects an image from the stored images, wherein the selected image forms a selected image (operation 1804). Next, the process labels the selected image with a label based on the classification of the abnormality for the selected image (operation 1806).

[0155] The process performs additional training of the machine learning model using the selected images with labels (operation 1808). The process then terminates.

[0156] Thus, these images can be used to train a machine learning model over time to identify anomalies for a particular fleet of aircraft. The process uses a feedback loop that can be performed periodically or continuously. The images can be for a fleet of aircraft operated by an organization, where additional training of the machine learning model increases the accuracy of the machine learning model in detecting anomalies for the fleet of aircraft. The organization can be selected from the group consisting of: airlines, governments, and aircraft manufacturers.

[0157] refer to Fig.19 , depicts an illustration of a flowchart of a process for selecting an image according to an illustrative embodiment. The process depicted in the flowchart is Fig.18An example of an implementation of operation 1804 in FIG.

[0158] The process selects images having anomalies identified in the fleet of aircraft, wherein the selected images form selected images wherein additional training of the machine learning model results in the machine learning model having increased accuracy in detecting anomalies in wiring harnesses for the fleet of aircraft (operation 1900). The process then terminates. Selecting images for anomalies in the fleet of aircraft may result in the machine learning model being fleet-specific to an airline or other organization. In one illustrative example, the fleet of aircraft may be selected from a group comprising: all aircraft operated by the organization, aircraft of the same type, aircraft on the same route, aircraft having the same cabin configuration, or aircraft within a life span.

[0159] Now go to Fig. 20 , depicts an illustration of a flowchart of a process for generating training data according to an illustrative embodiment. The process shown in this figure is a process that can be used Fig.17 Examples of additional operations performed by the operations in . This process can be used to generate synthetic data from computer-aided design models and real images of wire harnesses.

[0160] The process begins by generating a first synthetic wire harness image with anomalies using a computer-aided design model of the wire harness and anomalies defined by engineering standards (operation 2000). The process generates a second synthetic wire harness image with anomalies using a real image of the wire harness with anomalies (operation 2002).

[0161] The process classifies the first synthetic line beam image and the second synthetic line beam image, wherein the classification forms a label for the first synthetic line beam image and the second synthetic line beam image (operation 2004). The process adds the first synthetic line beam image and the second synthetic line beam image with the label to a training data set (operation 2006). The process then terminates.

[0162] Next, in Fig.21 In the figure, a flowchart of a process for generating training data is depicted according to an illustrative embodiment. The process shown in the figure is a process that can be used Fig.17 An example of additional operations performed by the operations in .

[0163] The process begins by creating a composite wire harness image using a second machine learning model that is trained to generate the composite wire harness image from wire harness images in a training data set (operation 2100). The process determines whether the composite wire harness image has an anomaly (operation 2102). The process adds a label to the composite wire harness image to indicate whether an anomaly exists (operation 2104). In this example, the composite wire harness image having an anomaly is a composite wiring anomaly image;

[0164] The process stores the synthesized wire harness images with the labels to create a synthetic training dataset (operation 2106 ). The process then terminates.

[0165] In the depicted example, the second machine learning model can be a generative artificial intelligence model. The model can be pre-trained in a generative adversarial network to create a synthetic wire harness image using the wire harness image.

[0166] exist Fig. 22 In the present invention, an illustration of a flowchart of a process for training a machine learning model for use in a mobile personal computing device is depicted according to an illustrative embodiment. In this illustrative example, the mobile personal computing device may have limited computing resources compared to a workstation or server computer.

[0167] The process trains a student machine learning model using the teacher machine learning model, wherein the student machine learning model is the machine learning model and is smaller in size than the teacher machine learning model, wherein the teacher machine learning model has been trained to detect anomalies in an image of a wire harness (operation 2200). The process then terminates.

[0168] Now go to Fig.23 , depicts an illustration of a flowchart of a process for performing remedial action according to an illustrative embodiment. The process shown in this figure is a process that can be used Fig.17 An example of additional operations performed by the operations in .

[0169] The process begins by identifying a remedial action in response to a result indicating that an anomaly exists in the wiring harness (operation 2300). In operation 2300, the remedial action can be identified in several different ways. For example, the machine learning model can include the remedial action. Therefore, the remedial action is identified based on the result returned from the machine learning model.

[0170] In another illustrative example, the indication of an anomaly in the result may also include a classification of the anomaly. The classification may be used to identify a remedial action.

[0171] In some cases, the classification can be based on the classification in the anomaly definition from the wire and cable damage and repair manual, and can include an anomaly code that can be used to identify a remedial action. By using the anomaly code identified according to the anomaly definition in the engineering standards manual, the remedial action can be determined based on the information from these engineering standards manuals.

[0172] The process performs a remedial action on the wire harness (operation 2302). The process then terminates. In operation 2302, a remedial action is performed on the actual physical wire harness in the aerospace vehicle. The remedial action may be, for example, repair, replace, log an anomaly, or some other suitable action.

[0173] The flow chart and block diagram in the embodiment of different descriptions show the framework, function and operation of some possible implementation methods of device and method in illustrative embodiment.In this regard, each block in flow chart or block diagram can represent at least one in module, fragment, function or a part of operation or step.For example, one or more in block can be implemented as program instruction, hardware or the combination of program instruction and hardware.When being implemented in hardware, hardware can for example take the form of integrated circuit, and this integrated circuit is manufactured or configured to perform one or more operations in flow chart or block diagram.When being implemented as the combination of program instruction and hardware, this embodiment can take the form of firmware.Each block in flow chart or block diagram can use special hardware system to implement, and this special hardware system performs the different operation or the combination of program instruction of special hardware and operation by special hardware.

[0174] In some alternative implementations of the illustrative embodiments, a function or functions shown in a block may not appear in the order shown in the figure. For example, in some cases, depending on the functions involved, two blocks shown in succession may be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. In addition, in addition to the blocks shown in the flow chart or block diagram, other blocks may be added.

[0175] Now go to Fig.24 , depicts a block diagram of a data processing system according to an illustrative embodiment. The data processing system 2400 may be used to implement Figure 1 104, server computer 106, or client device 110. In this illustrative example, data processing system 2400 includes communications framework 2402 that provides communications between processor unit 2404, memory 2406, persistent storage 2408, communication unit 2410, input / output (I / O) unit 2412, and display 2414. In this example, communications framework 2402 takes the form of a bus system.

[0176] Processor unit 2404 is used to execute instructions of software that can be loaded into memory 2406. Processor unit 2404 includes one or more processors. For example, processor unit 2404 can be selected from at least one of the following items: a multi-core processor, a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a network processor, or some other suitable type of processor. In addition, processor unit 2404 can be implemented using one or more heterogeneous processor systems, in which a main processor is present on a single chip with an auxiliary processor. As another illustrative example, processor unit 2404 may be a symmetric multi-processor system containing multiple processors of the same type on a single chip.

[0177] Memory 2406 and persistent storage 2408 are examples of storage devices 2416. A storage device is any piece of hardware capable of storing information (such as, for example, but not limited to, at least one of data, program instructions in functional form, or other suitable information) on a temporary basis, a permanent basis, or both a temporary basis and a permanent basis. In these illustrative examples, storage devices 2416 may also be referred to as computer-readable storage devices. In these examples, memory 2406 may be, for example, a random access memory or any other suitable volatile or non-volatile storage device. Persistent storage 2408 may take various forms, depending on the particular implementation.

[0178] For example, persistent storage 2408 may contain one or more components or devices. For example, persistent storage 2408 may be a hard drive, a solid state drive (SSD), flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination thereof. The media used by persistent storage 2408 may also be removable. For example, a removable hard drive may be used for persistent storage 2408.

[0179] In these illustrative examples, communications unit 2410 provides for communications with other data processing systems or devices. In these illustrative examples, communications unit 2410 is a network interface card.

[0180] Input / output unit 2412 allows for input and output of data with other devices that may be connected to data processing system 2400. For example, input / output unit 2412 may provide a connection for user input through at least one of a keyboard, a mouse, or some other suitable input device. Additionally, input / output unit 2412 may send output to a printer. Display 2414 provides a mechanism for displaying information to a user.

[0181] Instructions for at least one of an operating system, applications, or programs may be located in storage devices 2416, which are in communication with processor unit 2404 through communications framework 2402. The processes of the different embodiments may be performed by processor unit 2404 using computer-implemented instructions, which may be located in a memory, such as memory 2406.

[0182] These instructions are referred to as program instructions, computer usable program instructions, or computer readable program instructions, which can be read and executed by a processor in processor unit 2404. The program instructions in different embodiments may be embodied on different physical or computer readable storage media (such as memory 2406 or persistent storage device 2108).

[0183] Program instructions 2418 are located in functional form on computer readable media 2420 (which is selectively removable) and can be loaded onto or transferred to data processing system 2400 for execution by processor unit 2404. In these illustrative examples, program instructions 2418 and computer readable media 2420 form computer program product 2422. In the illustrative examples, computer readable media 2420 is computer readable storage media 2424.

[0184] Computer readable storage medium 2424 is a physical or tangible storage device for storing program instructions 2418, rather than a medium for propagating or transmitting program instructions 2418. Computer readable storage medium 2424 can be at least one of the following: electronic storage medium, magnetic storage medium, optical storage medium, electromagnetic storage medium, semiconductor storage medium, mechanical storage medium, or other physical storage medium. Some known types of storage devices that include these media include: floppy disk (diskette), hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device (such as punch card or pits / bosses formed in the main surface of the disk), or any suitable combination thereof.

[0185] Computer-readable storage medium 2424 (as that term is used in this disclosure) should not be construed as storing in the form of a transitory signal itself (such as at least one of a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide, an optical pulse through a fiber optic cable, an electrical signal transmitted through a wire, or other transmission medium).

[0186] In addition, during the normal operation of the storage device, data may be moved at some occasional point in time. These normal operations include access, defragmentation, or garbage collection. However, these operations do not present the storage device as transient because data is not transient when it is stored in the storage device.

[0187] Alternatively, the program instructions 2418 may be transferred to the data processing system 2400 using a computer readable signal medium. A computer readable signal medium is a signal that may be, for example, a propagated data signal containing the program instructions 2418. For example, a computer readable signal medium may be at least one of an electromagnetic signal, an optical signal, or any other suitable type of signal. These signals may be transmitted via a connection such as a wireless connection, a fiber optic cable, a coaxial cable, a wire, or any other suitable type of connection.

[0188] In addition, as used herein, "computer-readable medium 2420" may be singular or plural. For example, program instructions 2418 may be located in a computer-readable medium 2420 in the form of a single storage device or system. In another example, program instructions 2418 may be located in a computer-readable medium 2420 distributed in multiple data processing systems. In other words, some instructions in program instructions 2418 may be located in one data processing system, while other instructions in program instructions 2418 may be located in one data processing system. For example, a portion of program instructions 2118 may be located in a computer-readable medium 2420 in a server computer, while another portion of program instructions 2418 may be located in a computer-readable medium 2420 located in a group of client computers.

[0189] The different components illustrated for data processing system 2400 are not meant to provide architectural limitations to the manner in which the different embodiments may be implemented. In some illustrative examples, one or more of the components may be incorporated into, or otherwise form a part of, another component. For example, in some illustrative examples, memory 2406, or portions thereof, may be incorporated into processor unit 2404. The different illustrative embodiments may be implemented in a data processing system that includes components in addition to or in place of those illustrated for data processing system 2400. Fig.24 Other components shown in the can be different from the illustrative examples shown. The different embodiments can be implemented using any hardware device or system capable of running program instructions 2418.

[0190] Illustrative embodiments of the present disclosure may be used in Fig.25 Aircraft manufacturing and service method 2500 is shown and described in detail. Fig.26 The description will be made in the context of the illustrated aircraft 2600. Fig.25 , a diagram of an aircraft manufacturing and service method is depicted in accordance with an illustrative embodiment. During pre-production, aircraft manufacturing and service method 2500 may include Fig.26 Specification and design 2502 of aircraft 2600 and material procurement 2504.

[0191] During production, Fig.26 Component and subassembly manufacturing 2506 and system integration 2508 of aircraft 2600 occurs. Thereafter, Fig.26 The aircraft 2600 in may be certified and delivered 2510 for entry into service 2512. During the time period of customer entry into service 2512, Fig.26Aircraft 2600 in are scheduled for routine repair and maintenance 2514, which may include modification, reconfiguration, refurbishment, and other repair or maintenance.

[0192] Each of the processes of aircraft manufacturing and service method 2500 can be performed or carried out by a system integrator, a third party, an operator, or some combination thereof. In these examples, the operator can be a customer. For purposes of this description, a system integrator can include, but is not limited to, any number of aircraft manufacturers and major system subcontractors; a third party can include, but is not limited to, any number of vendors, subcontractors, and suppliers; and an operator can be an airline, a leasing company, a military entity, a maintenance organization, etc.

[0193] Reference now Fig.26 , depicts a diagram of an aircraft in which an illustrative embodiment may be implemented. In this example, aircraft 2600 is a Fig.25 2600 and may include airframe 2602 having multiple systems 2604 and interior 2606. Examples of systems 2604 include one or more of propulsion system 2608, electrical system 2610, hydraulic system 2612, and environmental system 2614. Any number of other systems may be included. Although an aerospace example is shown, different illustrative embodiments may be applied to other industries, such as the automotive industry.

[0194] The apparatus and methods embodied herein may be used in Fig.25 The method is employed during at least one of the stages of aircraft manufacturing and service method 2500.

[0195] In an illustrative example, Fig.25 Components and Subassemblies Manufacturing 2506 The components or subassemblies produced in 2506 may be manufactured in a manner similar to Fig.25 The method may be fabricated or manufactured in the same manner as parts or subassemblies produced when the aircraft 2600 is put into service 2512. As another example, one or more apparatus embodiments, method embodiments, or a combination thereof may be manufactured or manufactured during a production phase (such as Fig.25 One or more apparatus embodiments, method embodiments, or a combination thereof may be utilized during component and subassembly manufacturing 2506 and system integration 2508 of the aircraft 2600. Fig.25 The use of several different illustrative embodiments may substantially speed up the assembly of aircraft 2600, reduce the cost of aircraft 2600, or both speed up the assembly of aircraft 2600 and reduce the cost of aircraft 2600.

[0196] For example, Figure 2Wire harness inspection system 202 in may be used during at least one of component and subassembly manufacturing 2506 or system integration 2508 to inspect wire harnesses as they are manufactured and integrated into aircraft 2600. In another example, the wire harness inspection system may also be used to inspect wire harnesses during repair and maintenance 2514. Repair and maintenance 2514 may be routine repair and maintenance, which may include modification, reconfiguration, refurbishment, and other repair or maintenance.

[0197] Some features of the illustrative examples are described in the following clauses. These clauses are examples of features and are not intended to limit other illustrative examples.

[0198] Article 1

[0199] An aerospace wire harness inspection system, comprising:

[0200] Mobile personal computing devices;

[0201] a camera system connected to the mobile personal computing device;

[0202] a machine learning model running in the mobile personal computing device, wherein the machine learning model is trained to detect anomalies in a set of images of wiring harnesses in an aerospace vehicle; and

[0203] An inspection manager, the inspection manager being configured to:

[0204] controlling the camera system to generate a set of images of the harness;

[0205] sending the set of images of the wiring harness to the machine learning model; and

[0206] A result is received from the machine learning model indicating whether an anomaly exists in the wiring harness.

[0207] Article 2

[0208] The aerospace wire harness inspection system of clause 1, further comprising:

[0209] A computer system, the computer system being configured to:

[0210] storing images captured by a set of mobile personal computing devices, wherein storing the images captured from the set of mobile personal computing devices forms stored images;

[0211] storing, in association with the stored image, a classification of the anomaly identified in the image by the set of mobile personal computing devices;

[0212] selecting an image from said stored images, wherein selecting said image forms a selected image;

[0213] labeling the selected image with a label based on the classification of the anomaly for the selected image; and

[0214] Additional training of the machine learning model is performed using the selected images with the labels.

[0215] Article 3

[0216] An aerospace wire harness inspection system as described in claim 2, wherein the images are for a fleet of aircraft operated by an organization, and wherein the additional training of the machine learning model increases the accuracy of the machine learning model in detecting the anomalies for the fleet of aircraft.

[0217] Article 4

[0218] The aerospace wire harness inspection system of clause 3, wherein the organization is selected from the group consisting of: an airline, a government, and an aircraft manufacturer.

[0219] Article 5

[0220] An aerospace wire harness inspection system according to one of clauses 1 or 2, wherein, when selecting the image, the computer system is configured to:

[0221] Selecting an image having the anomaly identified in the fleet of aircraft, wherein selecting the image forms a selected image, wherein the additional training of the machine learning model results in the machine learning model having increased accuracy in detecting the anomaly in wiring harnesses for the fleet of aircraft.

[0222] Article 6

[0223] An aerospace wire harness inspection system according to clause 5, wherein the fleet of aircraft is selected from the group consisting of: all aircraft operated by an organization, aircraft of the same type, aircraft on the same route, aircraft with the same cabin configuration, or aircraft within a life cycle.

[0224] Article 7

[0225] The aerospace wire harness inspection system of one of clauses 1, 2, 3, 4, 5 or 6, further comprising:

[0226] A computer system, the computer system being configured to:

[0227] generating a first composite wire harness image having the anomaly using a computer-aided design model of the wire harness and the anomaly defined by engineering standards;

[0228] generating a second synthetic wire beam image having the anomaly using a real image of the wire beam having the anomaly;

[0229] classifying the first synthetic line beam image and the second synthetic line beam image, wherein the classification forms labels for the first synthetic line beam image and the second synthetic line beam image;

[0230] adding the first synthetic line beam image and the second synthetic line beam image having the label to a training dataset; and

[0231] The machine learning model is trained using the synthetic training dataset.

[0232] Article 8

[0233] The aerospace wire harness inspection system of one of clauses 1, 2, 3, 4, 5, 6 or 7, further comprising:

[0234] A computer system, the computer system being configured to:

[0235] creating a synthetic wire harness image using a second machine learning model, the second machine learning model being trained to generate the synthetic wire harness image from wire harness images in a training dataset;

[0236] determining whether the composite line beam image has the anomaly;

[0237] adding a label to the synthetic wiring harness image to indicate whether the anomaly exists, wherein the synthetic wiring harness image having the anomaly is a synthetic wiring anomaly image;

[0238] storing the synthetic wire harness images with the labels to create a synthetic training dataset; and training the machine learning model using the synthetic training dataset.

[0239] Article 9

[0240] An aerospace wire harness inspection system as described in claim 8, wherein the second machine learning model is a generative artificial intelligence model, which is trained in a generative adversarial network to create the synthetic wire harness image using a wire harness image.

[0241] Article 10

[0242] The aerospace wire harness inspection system of one of clauses 1, 2, 3, 4, 5, 6, 7, 8 or 9, further comprising:

[0243] a teacher machine learning model that has been trained to detect the anomaly in the image of a wire harness; and

[0244] A computer system, the computer system being configured to:

[0245] Using the teacher machine learning model to train a student machine learning model, wherein the student machine learning model is the machine learning model and is smaller in size than the teacher machine learning model.

[0246] Article 11

[0247] An aerospace wire harness inspection system as described in one of clauses 1, 2, 3, 4, 5, 6, 7, 8, 9 or 10, wherein the mobile personal computing device is selected from the group consisting of an extension pole, a handheld scanner, a smart phone, a mobile phone, a tablet computer, smart glasses, augmented reality goggles, a laptop computer and a smart watch.

[0248] Article 12

[0249] An aerospace wire harness inspection system according to one of clauses 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 or 11, wherein the aerospace vehicle is selected from the group consisting of: an aircraft, a commercial aircraft, a rotorcraft, a tiltrotor aircraft, a tiltrotor aircraft, a vertical take-off and landing aircraft, an electric vertical take-off and landing vehicle, a personal aerial vehicle, a spacecraft, a jet aircraft and a space shuttle.

[0250] Article 13

[0251] A platform wiring inspection system, comprising:

[0252] Mobile personal computing devices;

[0253] a camera system connected to the mobile personal computing device;

[0254] a machine learning model running in the mobile personal computing device, wherein the machine learning model is trained to detect anomalies in an image of a harness of a platform; and

[0255] An inspection manager, the inspection manager being configured to:

[0256] controlling the camera system to generate a set of images of the wire harness;

[0257] sending the set of images of the wiring harness to the machine learning model; and

[0258] A result is received from the machine learning model indicating whether an anomaly exists in the wiring harness.

[0259] Article 14

[0260] A platform wiring inspection system according to clause 13, wherein the platform is selected from the group consisting of: a mobile platform, a fixed platform, a land-based structure, a water-based structure, a space-based structure, an aircraft, a commercial aircraft, a rotorcraft, a tilt-rotor aircraft, a tilt-rotor aircraft, a vertical take-off and landing aircraft, an electric vertical take-off and landing vehicle, a personal aerial vehicle, a surface vessel, a tank, a personnel transport vehicle, a train, a spacecraft, a space station, a satellite, a submarine, a car, a power plant, a bridge, a dam, a house, a manufacturing facility and a building.

[0261] Article 15

[0262] A method for inspecting a wiring harness in an aerospace vehicle, the method comprising:

[0263] controlling a camera system via a computer system to generate a set of images of the wiring harness in the aerospace vehicle, wherein the camera system is connected to a mobile personal computing device;

[0264] sending, by the computer system, the set of images of the wiring harness to a machine learning model, wherein the machine learning model is trained to detect anomalies in the set of images of the wiring harness in the aerospace vehicle; and

[0265] A result is received, by the computer system, from the machine learning model indicating whether an anomaly exists in the wiring harness in the aerospace vehicle.

[0266] Article 16

[0267] The method of clause 15, further comprising:

[0268] storing, by said computer system, images captured by a set of mobile personal computing devices, wherein storing said images captured from the set of mobile personal computing devices forms stored images;

[0269] storing, by the computer system, in association with the stored image, the classification of the anomaly identified in the image by the set of mobile personal computing devices;

[0270] selecting, by the computer system, an image from the stored images, wherein selecting the image forms a selected image;

[0271] labeling, by the computer system, the selected image with a label based on the classification of the abnormality for the selected image; and

[0272] Additional training of the machine learning model is performed by the computer system using the selected images with the labels.

[0273] Article 17

[0274] The method of clause 16, wherein the images are of a fleet of aircraft operated by an organization, wherein the additional training of the machine learning model increases the accuracy of the machine learning model in detecting the anomalies for the fleet of aircraft.

[0275] Article 18

[0276] The method of clause 17, wherein the organization is selected from the group consisting of: an airline, a government, and an aircraft manufacturer.

[0277] Article 19

[0278] A method according to clause 16, 17 or 18, wherein selecting the image comprises:

[0279] Selecting, by the computer system, images having the anomalies identified in the fleet of aircraft, wherein selecting the images forms selected images, wherein the additional training of the machine learning model results in the machine learning model having increased accuracy in detecting the anomalies in wiring harnesses for the fleet of aircraft.

[0280] Article 20

[0281] A method according to clause 19, wherein the fleet of aircraft is selected from the group consisting of: all aircraft operated by the organization, aircraft of the same type, aircraft on the same route, aircraft with the same cabin configuration, or aircraft within a life span.

[0282] Article 21

[0283] A method according to one of clauses 15, 16, 17, 18, 19 or 20, further comprising:

[0284] generating, by the computer system, a first composite wire harness image having the anomaly using a computer-aided design model of the wire harness and the anomaly defined by engineering standards;

[0285] generating, by the computer system, a second synthetic wire beam image having the anomaly using the real image of the wire beam having the anomaly;

[0286] classifying, by the computer system, the first and second composite line beam images, wherein the classification forms labels for the first and second composite line beam images; and

[0287] The first and second synthetic line beam images having the labels are added to a training data set by the computer system.

[0288] Article 22

[0289] A method according to one of clauses 15, 16, 17, 18, 19, 20 or 21, further comprising:

[0290] creating, by the computer system, a synthetic line beam image using a second machine learning model, the second machine learning model being trained to generate the synthetic line beam image from line beam images in a training data set;

[0291] determining, by the computer system, whether the composite wire beam image has the anomaly;

[0292] adding, by the computer system, a label to the synthetic wiring harness image to indicate whether the anomaly exists, wherein the synthetic wiring harness image having the anomaly is a synthetic wiring anomaly image; and

[0293] The synthetic wire harness images with the labels are stored by the computer system to create a synthetic training data set.

[0294] Article 23

[0295] A method according to clause 22, wherein the second machine learning model is a generative artificial intelligence model, which is trained in a generative adversarial network to create the synthetic wire harness image using a wire harness image.

[0296] Article 24

[0297] A method according to one of clauses 15, 16, 17, 18, 19, 20, 21, 22 or 23, further comprising:

[0298] Using a teacher machine learning model to train a student machine learning model, wherein the student machine learning model is the machine learning model and is smaller in size than the teacher machine learning model, wherein the teacher machine learning model has been trained to detect the anomaly in the image of the wire harness.

[0299] Article 25

[0300] A method according to one of clauses 15, 16, 17, 18, 19, 20, 21, 22, 23 or 24, further comprising:

[0301] identifying, by the computer system, a remedial action in response to the result indicating the presence of the anomaly in the wiring harness; and

[0302] The remedial action is performed on the wiring harness.

[0303] Article 26

[0304] A method as described in one of clauses 15, 16, 17, 18, 19, 20, 21, 22, 23, 24 or 25, wherein the mobile personal computing device is selected from the group consisting of: an extension bar, a handheld scanner, a smart phone, a mobile phone, a tablet computer, smart glasses, augmented reality goggles, a laptop computer and a smart watch.

[0305] Article 27

[0306] A method according to one of clauses 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25 or 26, wherein the aerospace vehicle is selected from the group consisting of: an aircraft, a commercial aircraft, a rotorcraft, a tiltrotor aircraft, a tiltrotor aircraft, a vertical take-off and landing aircraft, an electric vertical take-off and landing vehicle, a personal aerial vehicle, a spacecraft, a jet aircraft and a space shuttle.

[0307] Thus, different illustrative examples provide methods, apparatus, systems, and computer program products that can be used to inspect wire harnesses and platforms (e.g., aerospace vehicles). In one illustrative example, an aerospace wire harness inspection system includes a mobile personal computing device, a camera system connected to the mobile personal computing device, a machine learning model running in the mobile personal computing device, and a controller. The machine learning model is trained to detect anomalies in a set of images of a wire harness in an aerospace vehicle. The controller, which is configured to control the camera system to generate a set of images of the wire harness, sends the set of images of the wire harness to the machine learning model, and receives a result from the machine learning model indicating whether an anomaly exists in the wire harness.

[0308] Use of an aerospace harness inspection system may result in more efficient identification of anomalies in harnesses compared to current techniques. Use of a machine learning model with a specialized training data set and training operations for training a machine learning model focused on harnesses for aerospace vehicles may result in increased efficiency in identifying anomalies. In various illustrative examples, a training data set may be generated in a manner that provides an increased amount of training data for training a machine learning model to identify harness anomalies.

[0309] Furthermore, by using a feedback loop and a continuously trained machine learning model, the aerospace harness inspection system can have an increasing accuracy in identifying anomalies. This increasing accuracy can be focused on a specific fleet of aircraft by using the system on these specific aircraft leads. Furthermore, by continuing to train the machine learning model based on images from these fleets of aircraft, the machine learning model can more efficiently and accurately identify anomalies that occur over time that are specific to these fleets of aircraft.

[0310] Additionally, the aerospace harness inspection system can be implemented in mobile personal computing devices such as handheld scanners, mobile phones, smart glasses, and other devices. In these illustrative examples, machine learning models can be created for use in these mobile personal computing devices so that the mobile computing devices can act as standalone devices without the need to send images to another location for analysis.

[0311] The descriptions of different illustrative embodiments have been presented for the purpose of illustration and description and are not intended to be exhaustive or limited to the embodiments of the disclosed forms. Different illustrative examples describe components that perform actions or operations. In the illustrative embodiments, the components can be configured to perform the described actions or operations. For example, the component can have a configuration or design for a structure that provides the component with the ability to perform the actions or operations described as being performed by the component in the illustrative examples. In addition, to the extent that the terms "include," "comprise," "have," "contain," and variations thereof are used herein, such terms are intended to be inclusive in a manner similar to the term "comprises" as an open transition word, without excluding any additional or other elements.

[0312] Many modifications and variations will be apparent to those of ordinary skill in the art. In addition, different illustrative embodiments may provide different features than other desired embodiments. The selected embodiment or embodiments are chosen and described in order to best explain the principles of the embodiments, the practical application, and to enable others of ordinary skill in the art to understand the disclosure of various embodiments with various modifications as are suitable for the particular use contemplated.

Claims

1. An aerospace wire harness inspection system (211), comprising: Mobile personal computing device (240); a camera system (213), the camera system (213) connected to the mobile personal computing device (240); a machine learning model (132, 224, 400, 500, 804, 1005, 1300) running in the mobile personal computing device (240), wherein the machine learning model (132, 224, 400, 500, 804, 1005, 1300) is trained to detect anomalies (205) in a set of images (129, 220) of wiring harnesses (222, 303) in an aerospace vehicle (206); and An inspection manager (214), the inspection manager (214) being configured to: controlling the camera system (213) to generate a set of images (129, 220) of the harness (222, 303); sending the set of images (129, 220) of the harness (222, 303) to the machine learning model (132, 224, 400, 500, 804, 1005, 1300); and A result (226) is received from the machine learning model (132, 224, 400, 500, 804, 1005, 1300) indicating whether an anomaly is present in the wiring harness (222, 303).

2. The aerospace wire harness inspection system (211) of claim 1, further comprising: A computer system (212), wherein the computer system (212) is configured to: storing images (129, 220) captured by a set of mobile personal computing devices, wherein storing the images (129, 220) captured from the set of mobile personal computing devices forms stored images; storing, in association with the stored image, a classification of the anomaly (205) identified in the image (129, 220) by the set of mobile personal computing devices; selecting an image (129, 220) from said stored images, wherein selecting said image (129, 220) forms a selected image; labeling the selected image with a label based on the classification of the anomaly (205) for the selected image; as well as Additional training of the machine learning model (132, 224, 400, 500, 804, 1005, 1300) is performed using the selected images with the labels.

3. The aerospace wire harness inspection system (211) of claim 2, wherein: The images (129, 220) are of a fleet of aircraft operated by an organization, wherein the additional training of the machine learning model (132, 224, 400, 500, 804, 1005, 1300) increases the accuracy of the machine learning model (132, 224, 400, 500, 804, 1005, 1300) in detecting the anomaly (205) for the fleet of aircraft.

4. The aerospace wire harness inspection system (211) of claim 2, wherein: Upon selecting the image (129, 220), the computer system (212) is configured to: selecting an image (129, 220) having the anomaly (205) identified in a fleet of aircraft, wherein selecting the image (129, 220) forms a selected image, wherein the additional training of the machine learning model (132, 224, 400, 500, 804, 1005, 1300) results in the machine learning model (132, 224, 400, 500, 804, 1005, 1300) having increased accuracy in detecting the anomaly (205) in wiring harnesses (203) for the fleet of aircraft.

5. The aerospace wire harness inspection system (211) of claim 4, wherein: The fleet of aircraft is selected from the group consisting of: all aircraft operated by the organization, aircraft of the same type, aircraft on the same route, aircraft with the same cabin configuration, or aircraft within a life span.

6. The aerospace wire harness inspection system (211) of claim 1, further comprising: A computer system (212), wherein the computer system (212) is configured to: generating a first composite wire harness image having the anomaly (205) using a computer-aided design model of the wire harness and the anomaly (205) defined by engineering standards; generating a second synthetic image of the wire beam having the anomaly (205) using a real image (129, 220) of the wire beam having the anomaly (205); classifying the first synthetic line beam image and the second synthetic line beam image, wherein the classification forms labels for the first synthetic line beam image and the second synthetic line beam image; as well as adding the first synthetic line beam image and the second synthetic line beam image with the label to a training dataset (506); as well as The training data set (506) is used to train the machine learning model (132, 224, 400, 500, 804, 1005, 1300).

7. The aerospace wire harness inspection system (211) of claim 1, further comprising: A computer system (212), wherein the computer system (212) is configured to: creating a synthetic line beam image using a second machine learning model (132, 224, 400, 500, 804, 1005, 1300), the second machine learning model (132, 224, 400, 500, 804, 1005, 1300) being trained to generate the synthetic line beam image from line beam images (129, 220) in a training data set (506); determining whether the composite line beam image has the anomaly (205); adding a tag to the composite wiring harness image to indicate whether the anomaly (205) is present, wherein the composite wiring harness image (525, 831) having the anomaly (527) is a composite wiring anomaly image (806); storing the synthetic wire harness images (525, 831) with the labels to create a synthetic training dataset (506); as well as The training data set (506) is used to train the machine learning model (132, 224, 400, 500, 804, 1005, 1300).

8. The aerospace wire harness inspection system (211) of claim 7, wherein: The second machine learning model (132, 224, 400, 500, 804, 1005, 1300) is a generative artificial intelligence model, which is trained in a generative adversarial network to create the synthetic wire beam image using the wire beam image (129, 220).

9. The aerospace wire harness inspection system (211) of claim 1, further comprising: a teacher machine learning model (132, 224, 400, 500, 804, 1005, 1300), said teacher machine learning model (132, 224, 400, 500, 804, 1005, 1300) having been trained to detect said anomaly (205) in said image (129, 220) of a wire harness; and A computer system (212), wherein the computer system (212) is configured to: Using the teacher machine learning model (132, 224, 400, 500, 804, 1005, 1300) to train a student machine learning model (132, 224, 400, 500, 804, 1005, 1300), wherein the student machine learning model (132, 224, 400, 500, 804, 1005, 1300) is the machine learning model (132, 224, 400, 500, 804, 1005, 1300) and is smaller in size than the teacher machine learning model (132, 224, 400, 500, 804, 1005, 1300).

10. The aerospace wire harness inspection system (211) of claim 1, wherein: The mobile personal computing device (240) is selected from the group consisting of: a handheld scanner, a smart phone, a smart watch, a laptop computer, and smart glasses.

11. The aerospace wire harness inspection system (211) of claim 1, wherein: The aerospace vehicle (206) is selected from the group consisting of an aircraft, a commercial aircraft, a rotorcraft, a tiltrotor aircraft, a tiltrotor aircraft, a vertical take-off and landing aircraft, an electric vertical take-off and landing vehicle, a personal aerial vehicle, a spacecraft, a jet aircraft, and a space shuttle.

12. A platform wiring inspection system, comprising: Mobile personal computing device (240); a camera system (213), the camera system (213) connected to the mobile personal computing device (240); a machine learning model (132, 224, 400, 500, 804, 1005, 1300) running in the mobile personal computing device (240), wherein the machine learning model (132, 224, 400, 500, 804, 1005, 1300) is trained to detect anomalies (205) in an image (129, 220) of a harness of a platform; and An inspection manager (214), the inspection manager (214) being configured to: controlling the camera system (213) to generate a set of images (129, 220) of the harness; sending the set of images (129, 220) of the harness to the machine learning model (132, 224, 400, 500, 804, 1005, 1300); and A result (226) is received from the machine learning model (132, 224, 400, 500, 804, 1005, 1300) indicating whether an anomaly exists in the wiring harness.

13. The platform wiring inspection system according to claim 12, wherein: The platform is selected from the group consisting of a mobile platform, a fixed platform, a land-based structure, a water-based structure, a space-based structure, an aircraft, a commercial aircraft, a rotorcraft, a tiltrotor aircraft, a tiltrotor aircraft, a vertical take-off and landing aircraft, an electric vertical take-off and landing vehicle, a personal aerial vehicle, a surface vessel, a tank, a personnel transport vehicle, a train, a spacecraft, a space station, a satellite, a submarine, a car, a power plant, a bridge, a dam, a house, a manufacturing facility and a building.

14. A method for inspecting a wiring harness in an aerospace vehicle (206), the method comprising: controlling (1700) a camera system (213) via a computer system (212) to generate a set of images (129, 220) of the harness in the aerospace vehicle (206), wherein the camera system (213) is connected to a mobile personal computing device (240); sending (1702) the set of images (129, 220) of the wiring harness to a machine learning model (132, 224, 400, 500, 804, 1005, 1300) via the computer system (212), wherein the machine learning model (132, 224, 400, 500, 804, 1005, 1300) is trained to detect anomalies (205) in the set of images (129, 220) of the wiring harness in the aerospace vehicle (206); and A result (226) is received (1704) by the computer system (212) from the machine learning model (132, 224, 400, 500, 804, 1005, 1300) indicating whether an anomaly exists in the wiring harness in the aerospace vehicle (206).

15. The method according to claim 14, further comprising: storing (1800) by the computer system (212) images (129, 220) captured by a set of mobile personal computing devices, wherein storing the images (129, 220) captured from the set of mobile personal computing devices forms stored images; storing (1802), by the computer system (212), in association with the stored image, a classification of the anomaly (205) identified in the image (129, 220) by the set of mobile personal computing devices; selecting (1804) an image (129, 220) from the stored images by the computer system (212), wherein selecting the image (129, 220) forms a selected image; labeling (1806), by the computer system (212), the selected image with a label based on the classification of the anomaly (205) for the selected image; and Additional training of the machine learning model (132, 224, 400, 500, 804, 1005, 1300) is performed (1808) by the computer system (212) using the selected images with the labels.

16. The method according to claim 15, wherein: Selecting the image (129, 220) includes: Selecting (1900), by the computer system (212), an image (129, 220) having the anomaly (205) identified in the fleet of aircraft, wherein selecting the image (129, 220) forms a selected image, wherein the additional training of the machine learning model (132, 224, 400, 500, 804, 1005, 1300) results in the machine learning model (132, 224, 400, 500, 804, 1005, 1300) having increased accuracy in detecting the anomaly (205) in wiring harnesses for the fleet of aircraft.

17. The method according to claim 16, wherein: The fleet of aircraft is selected from the group consisting of: all aircraft operated by the organization, aircraft of the same type, aircraft on the same route, aircraft with the same cabin configuration, or aircraft within a life span.

18. The method according to claim 14, further comprising: generating (2000) by the computer system (212) a first composite wire harness image having the anomaly (205) using a computer-aided design model of the wire harness and the anomaly (205) defined by engineering standards; generating (2002) a second synthetic wire harness image having the anomaly (205) by the computer system (212) using the real image (129, 220) of the wire harness having the anomaly (205); classifying (2004) the first synthetic line beam image and the second synthetic line beam image by the computer system (212), wherein the classification forms labels for the first synthetic line beam image and the second synthetic line beam image; as well as The first and second synthetic line beam images having the labels are added (2006) to a training data set (506) by the computer system (212).

19. The method according to claim 14, further comprising: creating (2100) a synthetic line beam image by the computer system (212) using a second machine learning model (132, 224, 400, 500, 804, 1005, 1300), the second machine learning model (132, 224, 400, 500, 804, 1005, 1300) being trained to generate the synthetic line beam image from line beam images (129, 220) in a training data set (506); determining (2102) by the computer system (212) whether the composite line beam image has the anomaly (205); adding (2104) a tag to the composite wiring harness image by the computer system (212) to indicate whether the anomaly (205) is present, wherein the composite wiring harness image (525, 831) having the anomaly (525, 833) is a composite wiring anomaly image (806); and The synthetic wire harness images with the labels are stored (2106) by the computer system (212) to create a synthetic training data set (506).

20. The method of claim 14, further comprising: Using a teacher machine learning model (132, 224, 400, 500, 804, 1005, 1300) to train (2200) a student machine learning model (132, 224, 400, 500, 804, 1005, 1300), wherein the student machine learning model (132, 224, 400, 500, 804, 1005, 1300) is a 00, 500, 804, 1005, 1300) and is smaller in size than the teacher machine learning model (132, 224, 400, 500, 804, 1005, 1300), wherein the teacher machine learning model (132, 224, 400, 500, 804, 1005, 1300) has been trained to detect the anomaly (205) in the image (129, 220) of the wire harness.

21. The method of claim 14, further comprising: identifying (2300), by the computer system (212), a remedial action in response to the result (226) indicating the presence of the anomaly in the wiring harness; as well as The remedial action is performed (2302) on the wiring harness.

Images