Manufacturing system change classification

Abstract

This application relates to manufacturing system change classification. A computing system (10) is provided, comprising one or more processing devices (12) configured to receive a manufacturing change specification (20) of a proposed change (22) to a manufacturing system (24). At a first trained machine learning model (30), the one or more processing devices (12) are further configured to calculate one or more predicted change categories (32) of the proposed change (22). At a second trained machine learning model (40), the one or more processing devices (12) are further configured to calculate a change magnitude classification (42) associated with the proposed change (22). The second trained machine learning model (40) is configured to calculate the change magnitude classification (42) at least in part based on one or more predicted change categories (32). The one or more processing devices (12) are further configured to output one or more predicted change categories (32) and change magnitude classification (42) to a user interface (50).

Description

Technical Field

[0001] This disclosure relates to the development of manufacturing systems and processes. Background Technology

[0002] Aircraft manufacturing systems and processes are highly complex and can comprise thousands of different equipment components and manufacturing steps. Therefore, a change to one part of an aircraft manufacturing system or process can affect many other components or steps. Before making such changes, Design Licensing Engineers (DAEs) review them to determine their downstream impact. Reviewing aircraft manufacturing changes and checking dependencies is a time-consuming task that can sometimes slow down the entire development process of aircraft manufacturing systems and processes.

[0003] Aircraft manufacturing is also an area where changes to manufacturing systems and processes sometimes require regulatory approval. Due to the complexity of aircraft manufacturing systems and processes, determining whether a change requires regulatory approval can be a difficult and time-consuming task for DAEs. Summary of the Invention

[0004] According to one aspect of this disclosure, a computing system is provided, including one or more processing devices configured to receive a manufacturing change specification for a proposed change to a manufacturing system. At a first trained machine learning model, the one or more processing devices are further configured to calculate one or more predicted change categories for the proposed change. At a second trained machine learning model, the one or more processing devices are further configured to calculate a change magnitude classification associated with the proposed change. The second trained machine learning model is configured to calculate the change magnitude classification at least in part based on the one or more predicted change categories. The one or more processing devices are further configured to output the one or more predicted change categories and change magnitude classifications to a user interface. Attached Figure Description

[0005] Figure 1 An exemplary computing system is schematically illustrated according to an exemplary embodiment, in which one or more processing devices receive manufacturing change specifications.

[0006] Figure 2 It schematically shows the following based on Figure 1 An example of a computing system when one or more processing devices are configured to train a first trained machine learning model.

[0007] Figure 3 It schematically shows the following based on Figure 2 An example of a computing system when one or more processing devices are configured to train a second trained machine learning model.

[0008] Figure 4It shows according to Figure 1 Examples include data included in the manufacturing change specification and exemplary tables of data output by the first trained machine learning model.

[0009] Figure 5 It shows according to Figure 1 The example is a computational system that performs additional training on a first trained machine learning model and a second trained machine learning model.

[0010] Figure 6A It shows according to Figure 1 The flowchart illustrates an example of a method used in conjunction with a computing system to classify proposed changes to a manufacturing system.

[0011] Figure 6B It shows Figure 6A Additional steps in the method can be performed to train a first trained machine learning model and a second trained machine learning model.

[0012] Figure 6C It shows Figure 6A Additional steps in the method, which in some examples can be performed to update the machine learning model.

[0013] Figure 7 The illustration shows where instantiation is possible. Figure 1 An exemplary computing environment for a computing system. Detailed Implementation

[0014] To address the aforementioned challenges, a computing system 10 is provided, such as... Figure 1 As shown. Figure 1 An exemplary computing system 10 is schematically illustrated, comprising one or more processing devices 12 and one or more memory devices 14. For example, the one or more processing devices 12 may include one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more dedicated hardware accelerators. The one or more memory devices 14 may include volatile memory and / or non-volatile memory. The computing system 10 may be implemented as a single physical computing device or multiple communication-coupled physical computing devices, such as in a server-client configuration. Figure 1 The computing system 10 shown also includes one or more input devices 16 and one or more output devices 18, through which users can interact with the computing system 10.

[0015] like Figure 1As shown, one or more processing devices 12 are configured to receive a manufacturing change specification 20 for a proposed change 22 to the manufacturing system 24. For example, the proposed change 22 may be a proposed change to one or more manufacturing devices 25 included in the manufacturing system 24. Additionally or alternatively, the proposed change 22 may be a proposed change to one or more manufacturing steps 27 of a manufacturing process 26 performed at the manufacturing system 24. Figure 1 In this example, manufacturing system 24 is configured to manufacture aircraft 28. In this example, proposed change 22 could be a proposed change to one or more components 29 of aircraft 28 manufactured at manufacturing system 24.

[0016] At the first trained machine learning model 30, one or more processing devices 12 are also configured to compute one or more predicted change categories 32 for the proposed change 22. For example, one or more predicted change categories 32 may specify one or more manufacturing devices 25 included in the manufacturing system 24. As another example, one or more predicted change categories 32 may specify one or more manufacturing steps 27 of the manufacturing process 26. In an example where the manufacturing system 24 is configured to manufacture an aircraft 28, one or more predicted change categories 32 may specify one or more components 29 of the aircraft 28 manufactured at the manufacturing system 24.

[0017] The first trained machine learning model 30 is configured to receive manufacturing change specification 20 as input. Figure 1 The first trained machine learning model 30 shown in the example is a multi-label classifier model that classifies the proposed change 22 based on whether it belongs to each of a plurality of potential change categories. The first trained machine learning model 30 could be, for example, an XGBoost model. As another example, the first trained machine learning model 30 could be a random forest model. In some examples, the first trained machine learning model 30 outputs one or more predicted change categories as an n-dimensional vector of Boolean values, where n is the number of potential change categories.

[0018] Figure 2 The diagram schematically illustrates a computing system 10 configured to train a first trained machine learning model 30 using one or more processing devices 12. The first trained machine learning model 30 is trained using first training data 60, which includes multiple prior manufacturing change specifications 62. The multiple prior manufacturing change specifications 62 indicate corresponding prior changes to the manufacturing system 24. Furthermore, the first training data 60 includes corresponding multiple training change categories 64 for the prior changes. The training change categories 64 may be change categories previously selected by one or more DAEs that have reviewed the prior manufacturing change specifications 62.

[0019] In some examples, for each prior manufacturing change specification 62, one or more processing devices 12 are also configured to calculate one or more key phrases 66 associated with that prior manufacturing change specification 62. In these examples, the first training data 60 also includes the key phrases 66. Figure 2 As illustrated in the example, the key phrase 66 can be computed in the natural language processing (NLP) model 68, at least in part, by inputting the previous manufacturing change specification 62 into the NLP model 68. In examples where there are style differences and / or template differences among multiple previous manufacturing change specifications 62, including the key phrase 66 in the first training data 60 can make the output of the first trained machine learning model 30 more consistent.

[0020] In each of the multiple training iterations 78, one or more processing devices 12 are further configured to feed batches of previous manufacturing change specifications 62 and corresponding key phrases 66 into a first machine learning model 70. The first machine learning model 70 is configured to compute one or more corresponding training-time predicted change categories 72 for each previous manufacturing change specification 62. The one or more processing devices 12 are also configured to compute a corresponding value for a first loss function 74 based at least in part on the training-time predicted change categories 72 and the training change categories 64. For example, the first loss function 74 may be a Hamming loss function.

[0021] One or more processing devices 12 are further configured to compute a model update 76 based at least in part on the value of a first loss function 74. This model update 76 may be computed via stochastic gradient descent (SGD). The one or more processing devices 12 are also configured to apply the model update 76 to a first machine learning model 70. Thus, the one or more processing devices 12 are configured to train the first machine learning model 70 through multiple training iterations 78 to obtain a first trained machine learning model 30.

[0022] Back Figure 1 For example, one or more processing devices 12 are also configured to compute a change magnitude classification 42 associated with the proposed change 22 at a second trained machine learning model 40. In some examples, the change magnitude classification 42 may be a binary classification. In these examples, the change magnitude classification 42 may be a classification of whether the proposed change 22 is a major change (a change requiring regulatory approval) or a minor change (a change not requiring regulatory approval). In some examples, the change magnitude classification 42 is a multidimensional binary classification that indicates the majorness or minorness of each of multiple categories (e.g., proposed change category 32).

[0023] The second trained machine learning model 40 is configured to receive manufacturing change specification 20 as input. Furthermore, the second trained machine learning model 40 is configured to calculate a change magnitude classification 42 based at least in part on one or more predicted change categories 32 calculated for the proposed change 22 at the first trained machine learning model 30. Since changes to some manufacturing equipment 25, manufacturing steps 27, and / or parts 29 are more likely to be significant compared to other changes, the second trained machine learning model 40 can calculate the change magnitude classification 42 more accurately by using one or more predicted change categories 32 as input. The second trained machine learning model 40 could, for example, be an XGBoost model.

[0024] Figure 3 The computational system 10 is schematically illustrated during the training of a second trained machine learning model 40. One or more processing devices 12 are configured to train the second trained machine learning model 40 using second training data 80 that includes multiple previous manufacturing change specifications 62. The second training data 80 also includes multiple training change categories 64 for previous changes indicated in the previous manufacturing change specifications 62. In an example where one or more processing devices 12 are configured to compute a key phrase 66 associated with the previous manufacturing change specification 62, the key phrase 66 may also be included in the second training data 80. The second training data 80 also includes multiple corresponding previous change magnitude classifications 82 associated with the previous changes. For example, the previous change magnitude classification 82 may be a change magnitude classification assigned to the previous manufacturing change specification 62 by one or more DAEs.

[0025] In each of the plurality of training iterations 98, one or more processing devices 12 are further configured to feed batches of the previous manufacturing change specification 62, training change category 64, and key phrase 66 into a second machine learning model 90, which is configured to compute a corresponding training time change magnitude classification 92 associated with the previous manufacturing change specification 62. One or more processing devices 12 are also configured to compute the value of a second loss function 94 based at least in part on the training time change magnitude classification 92 and the previous change magnitude classification 82.

[0026] One or more processing devices 12 are also configured to compute a model update 96 (e.g., via SGD) at least in part based on a second loss function 94 and apply the model update 96 to a second machine learning model 90. Thus, one or more processing devices 12 are configured to update the second machine learning model 90 across multiple training iterations 98 to obtain a second trained machine learning model 40.

[0027] Back Figure 1For example, after calculating one or more predicted change categories 32 and change magnitude classifications 42 associated with the proposed change 22, one or more processing devices 12 are also configured to output one or more predicted change categories 32 and change magnitude classifications 42 to a user interface 50. For example, the user interface 50 may be a graphical user interface (GUI) displayed at one or more display devices included in one or more output devices 18 in the computing system 10.

[0028] Figure 4 An exemplary mapping is shown from data included in manufacturing change specifications 20 to predicted change categories 32 associated with these manufacturing change specifications 20. Figure 4 The first table 100 shown includes a corresponding change identifier associated with proposed change 22. The first table 100 also includes a "Payload Keyword Count" column, which shows the number of key phrases 66 associated with those manufacturing change specifications 20 related to the aircraft payload. The first table 100 also includes a "Change Type" column, which includes a corresponding change type label given to proposed change 22 in manufacturing change specification 20. Furthermore, the first table 100 includes an "ATA Section" column, which indicates the specific provisions listed in manufacturing change specification 20 related to proposed change 22.

[0029] Figure 4 A second table 102 is also shown, which includes a Boolean indicator (0 or 1) for the proposed change 22. The Boolean indicator for the proposed change 22 specifies whether the first trained machine learning model 30 has identified the proposed change 22 as belonging to each of the corresponding plurality of predicted change categories. Therefore, in Figure 4 In the example, the rows of the second table 102 are an n-dimensional vector of Boolean values ​​output by the first trained machine learning model 30. Figure 4 The columns of the second table 102 shown in the example are "Internal", "Electrical", "EME", "Cavity", "Aerodynamics", and "Avionics". The table below shows experimental data related to the performance of the first trained machine learning model 30. The table also compares the performance of various statistical models, including XGBoost (XGB), Random Forest (RF), k-Nearest Neighbors (KNN), Support Vector Machine (SVM), Decision Tree, and Logistic Representation. The above shows that XGB has the highest performance in recall and F1 score, and the second highest performance in Hamming loss and precision. The RF model has the highest performance in Hamming loss and precision, and the second highest performance in recall and F1 score.

[0030] The table below shows experimental data related to the performance of the second trained machine learning model 40. The table below compares the performance of various machine learning models, including weighted XGB, unweighted XGB, multilayer perceptron (MLP), AdaBoost, weighted support vector classifier (SVC), and balanced RF. The above shows that the unweighted XGB has the highest performance in accuracy, precision, and F1, and the second highest performance in recall.

[0031] Figure 5 The example shown is a computational system 10 performing additional training on a first trained machine learning model 30 and a second trained machine learning model 40 after deployment. Figure 5 In one example, one or more processing devices 12 are further configured to receive additional training data 114, which includes one or more ground reality change categories 110 and ground reality change magnitude classifications 112 associated with the proposed change 22. After one or more processing devices 12 output one or more predicted change categories 32 and change magnitude classifications 42, one or more ground reality change categories 110 and ground reality change magnitude classifications 112 are received via a user interface 50 as user feedback. The user can input one or more ground reality change categories 110 and ground reality change magnitude classifications 112 using one or more input devices 16. In some examples, one or more ground reality change categories 110 and ground reality change magnitude classifications 112 are permissions for one or more predicted change categories and change magnitude classifications 42 computed at a first trained machine learning model 30 and a second trained machine learning model 40, respectively.

[0032] One or more processing devices 12 are also configured to perform additional training using additional training data 114 on the first trained machine learning model 30 and the second trained machine learning model 40. Therefore, one or more processing devices 12 are configured to compute the first further trained machine learning model 116 and the second further trained machine learning model 118, which are updated to reflect one or more ground reality change categories 110 and ground reality change magnitude categories 112 selected by the user. (Refer to the above) Figures 2-3 The aforementioned additional training can be performed on the first trained machine learning model 30 and the second trained machine learning model 40.

[0033] Figure 6AA flowchart of a method 200 for classifying proposed changes to a manufacturing system in conjunction with a computing system is shown. In step 202, method 200 includes receiving a manufacturing change specification for the proposed changes to the manufacturing system. For example, a proposed change could be a change to one or more manufacturing facilities and / or one or more manufacturing steps. In some examples, the manufacturing system is configured to manufacture an aircraft. The proposed change could be a change to one or more components of the aircraft. The manufacturing change specification could be a document uploaded at a user interface.

[0034] In step 204, method 200 further includes computing one or more predicted change categories for the proposed change at the first trained machine learning model. For example, the first trained machine learning model could be an XGBoost model or a random forest model. In some examples, the first trained machine learning model outputs an n-dimensional vector of Boolean values, where n is the number of potential change categories. The one or more predicted change categories may specify one or more manufacturing devices included in the manufacturing system, one or more manufacturing steps in the manufacturing process, or one or more components of an aircraft or other equipment manufactured at the manufacturing system.

[0035] In step 206, method 200 further includes computing a change magnitude classification associated with the proposed change at a second trained machine learning model. The second trained machine learning model is configured to compute the change magnitude classification based at least in part on one or more predicted change categories. Furthermore, the second trained machine learning model is configured to receive a manufacturing change specification as input. In some examples, the change magnitude classification is a binary classification. For example, a binary classification could be a classification of whether the proposed change requires regulatory approval. In some examples, the change magnitude classification could be a multidimensional binary classification computed for multiple different categories, such as the predicted change categories computed in step 204.

[0036] In step 208, method 200 further includes outputting one or more predicted change categories and change magnitude classifications to the user interface. Therefore, the DAE can review the predicted change categories and change magnitude classifications made by the machine learning model.

[0037] Figure 6B Additional steps of method 200, which can be performed to train a first trained machine learning model and a second trained machine learning model, are shown. In step 210, method 200 includes training the first trained machine learning model using first training data that includes a plurality of previous manufacturing change specifications and corresponding plurality of training change categories. The previous manufacturing change specifications indicate corresponding previous changes to the manufacturing system, and the training change categories are associated with the previous changes.

[0038] In some examples, step 210 also includes step 212. In step 212, for each previous manufacturing change specification, method 200 further includes computing one or more key phrases associated with that previous manufacturing change specification at a natural language processing model. In examples of performing step 212, the first training data also includes the key phrases.

[0039] In step 214, method 200 further includes training a second trained machine learning model using second training data. The second training data includes multiple previous manufacturing change specifications included in the first training data. Furthermore, the second training data includes multiple training change categories for previous changes indicated in the previous manufacturing change specifications. The second training data also includes corresponding multiple previous change magnitude classifications associated with the previous changes. In the example of performing step 212, key phrases may also be included in the second training data.

[0040] Figure 6C Additional steps of method 200, which can be performed in some examples to update a machine learning model, are shown. In step 216, method 200 further includes receiving one or more ground reality change categories and ground reality change magnitude classifications via a user interface. After outputting one or more predicted change categories and change magnitude classifications, one or more ground reality change categories and ground reality change magnitude classifications are received.

[0041] In step 218, method 200 further includes performing additional training on the first trained machine learning model and the second trained machine learning model using additional training data that includes one or more ground reality change categories and ground reality change magnitude classifications. Therefore, user feedback can be used to update the first trained machine learning model and the second trained machine learning model.

[0042] Using the systems and methods described above, machine learning models are used to programmatically classify proposed changes to the manufacturing process. These machine learning models predict which parts of the manufacturing system or process will be affected by the proposed changes, and the magnitude of the changes. The experimental results show that the machine learning models achieve high accuracy in these predictions. Therefore, the systems and methods described above can significantly reduce the amount of time spent by DAEs in determining the impact of proposed changes to the manufacturing system.

[0043] The methods and processes described herein are associated with a computing system of one or more computing devices. In particular, such methods and processes may be implemented as computer applications or services, application programming interfaces (APIs), libraries, and / or other computer program products.

[0044] Figure 7A non-limiting embodiment of a computing system 300 capable of performing one or more of the methods and processes described above is schematically illustrated. The computing system 300 is shown in a simplified form. The computing system 300 can embody the above description and... Figure 1 The computing system 10 is illustrated in the figure. Components of the computing system 300 may be included in the form of one or more personal computers, server computers, tablet computers, home entertainment computers, network computing devices, mobile computing devices, mobile communication devices (e.g., smartphones) and / or other computing devices, as well as wearable computing devices (such as smartwatches and head-mounted augmented reality devices).

[0045] The computing system 300 includes a processing circuit system 302, volatile memory 304, and non-volatile storage device 306. The computing system 300 may optionally include a display subsystem 308, an input subsystem 310, a communication subsystem 312, and / or... Figure 7 Other components not shown.

[0046] Processing circuitry system 302 typically includes one or more logic processors, which are physical devices configured to execute instructions. For example, a logic processor may be configured to execute instructions that are part of one or more application programs, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform tasks, implement data types, change the state of one or more components, achieve technical effects, or otherwise achieve desired results.

[0047] A logical processor may include one or more physical processors configured to execute software instructions. Additionally or alternatively, a logical processor may include one or more hardware logic circuits or firmware devices configured to execute hardware-implemented logic or firmware instructions. The processor of the processing circuit system 302 is single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and / or distributed processing. Individual components of the processing circuit system 302 may optionally be distributed across two or more separate devices that are remotely located and / or configured for coordinated processing. For example, aspects of the computing system 300 described herein are virtualized and executed by remotely accessible networked computing devices configured in a cloud computing configuration. In such a case, it is understood that these virtualized aspects run on different physical logical processors on various different machines. These different physical logical processors on different machines will be understood to be entirely covered by the processing circuit system 302.

[0048] The non-volatile storage device 306 includes one or more physical devices configured to store instructions executable by a processing circuitry system to implement the methods and processes described herein. When such methods and processes are implemented, the state of the non-volatile storage device 306 can be transformed—for example, to store different data.

[0049] Non-volatile storage device 306 may include removable and / or built-in physical devices. Non-volatile storage device 306 may include optical memory, semiconductor memory and / or magnetic memory, or other mass storage technologies. Non-volatile storage device 306 may include non-volatile, dynamic, static, read / write, read-only, sequential access, location-addressable, file-addressable, and / or content-addressable devices. It should be understood that non-volatile storage device 306 is configured to retain instructions even when power to non-volatile storage device 306 is cut off.

[0050] Volatile memory 304 may include a physical device containing random access memory. Volatile memory 304 is typically used by the processing circuitry system 302 to temporarily store information during the processing of software instructions. It should be understood that when the power supply to volatile memory 304 is cut off, volatile memory 304 typically does not continue storing instructions.

[0051] Various aspects of the processing circuitry system 302, the volatile memory 304, and the non-volatile storage device 306 can be integrated into one or more hardware logic components. Such hardware logic components may include, for example, field-programmable gate arrays (FPGAs), application-specific integrated circuits (PASICs / ASICs), application-specific standard products (PSSPs / ASSPs), system-on-a-chip (SoCs), and complex programmable logic devices (CPLDs).

[0052] The terms "module," "program," and "engine" can be used to describe aspects of a computing system 300 typically implemented in software by a processor to perform specific functions using portions of volatile memory, involving transformative processing that specifically configures the processor to perform those functions. Thus, a module, program, or engine can be instantiated using portions of volatile memory 304, via processing circuitry 302, executing instructions stored in non-volatile storage device 306. It should be understood that different modules, programs, and / or engines can be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Similarly, different applications, services, code blocks, objects, routines, APIs, functions, etc., can instantiate the same modules, programs, and / or engines. The terms "module," "program," and "engine" can encompass individual or grouped executable files, data files, libraries, drives, scripts, database records, etc.

[0053] When included, the display subsystem 308 can be used to present a visual representation of the data stored by the non-volatile storage device 306. This visual representation may take the form of a graphical user interface (GUI). When the methods and processes described herein change the data stored in the non-volatile storage device and thus change the state of the non-volatile storage device, the state of the display subsystem 308 can also be changed to visually represent the change in the underlying data. The display subsystem 308 may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with the processing circuitry system 302, the volatile memory 304, and / or the non-volatile storage device 306 in a shared housing, or such display devices may be peripheral display devices.

[0054] When included, the input subsystem 310 may include or interface with one or more user input devices, such as a keyboard, mouse, touchscreen, camera, or microphone.

[0055] When included, the communication subsystem 312 can be configured to communicatively couple the various computing devices described herein to each other and to other devices. The communication subsystem 312 may include wired and / or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem can be configured to communicate via wired or wireless local area networks or wide area networks, broadband cellular networks, etc. In some embodiments, the communication subsystem may allow the computing system 300 to send messages to and / or receive messages from other devices via a network such as the Internet.

[0056] In addition, this disclosure includes configurations based on the following examples.

[0057] Example 1. A computing system comprising: one or more processing devices configured to: receive a manufacturing change specification for a proposed change to a manufacturing system; compute one or more predicted change categories of the proposed change at a first trained machine learning model; compute a change magnitude classification associated with the proposed change at a second trained machine learning model, wherein the second trained machine learning model is configured to compute the change magnitude classification at least in part based on the one or more predicted change categories; and output the one or more predicted change categories and the change magnitude classification to a user interface.

[0058] Example 2. The computing system according to Example 1, wherein the one or more processing devices are further configured to train the first trained machine learning model using first training data, the first training data comprising: a plurality of previous manufacturing change specifications indicating corresponding previous changes to the manufacturing system; and a plurality of corresponding training change categories for the previous changes.

[0059] Example 3. The computing system according to Example 2, wherein: for each of the previous manufacturing change specifications, the one or more processing devices are further configured to compute one or more key phrases associated with the previous manufacturing change specification at a natural language processing model; and the first training data further includes the key phrases.

[0060] Example 4. The computing system according to Example 2 or 3, wherein the one or more processing devices are further configured to train the second trained machine learning model using second training data, the second training data including: the plurality of previous manufacturing change specifications; the plurality of training change categories of the previous changes indicated in the previous manufacturing change specifications; and corresponding plurality of previous change magnitude classifications associated with the previous changes.

[0061] Example 5. A computing system according to any one of Examples 1-4, wherein the change category classification is a multidimensional binary classification.

[0062] Example 6. A computing system according to any one of Examples 1-5, wherein the one or more processing devices are further configured to: after outputting the one or more predicted change categories and the change magnitude classifications, receive one or more ground reality change categories and ground reality change magnitude classifications via the user interface; and perform additional training on the first trained machine learning model and the second trained machine learning model using additional training data including the one or more ground reality change categories and the ground reality change magnitude classifications.

[0063] Example 7. A computing system according to any one of Examples 1-6, wherein the one or more predicted change categories specify one or more manufacturing devices included in the manufacturing system.

[0064] Example 8. A computing system according to any one of Examples 1-7, wherein the one or more predicted change categories specify one or more manufacturing steps of a manufacturing process.

[0065] Example 9. A computing system according to Example 1, wherein: the manufacturing system is configured to manufacture an aircraft; and the one or more predicted change categories specify one or more components of the aircraft manufactured at the manufacturing system.

[0066] Example 10. A computational system according to any one of Examples 1-9, wherein the first trained machine learning model outputs an n-dimensional vector of Boolean values, where n is the number of potential change categories.

[0067] Example 11. A method for use with a computing system, the method comprising: receiving a manufacturing change specification of a proposed change to a manufacturing system; calculating one or more predicted change categories of the proposed change at a first trained machine learning model; calculating a change magnitude classification associated with the proposed change at a second trained machine learning model, wherein the second trained machine learning model is configured to calculate the change magnitude classification at least in part based on the one or more predicted change categories; and outputting the one or more predicted change categories and the change magnitude classification to a user interface.

[0068] Example 12. The method according to Example 11 further includes training the first trained machine learning model using first training data, the first training data including: a plurality of previous manufacturing change specifications indicating corresponding previous changes to the manufacturing system; and a plurality of corresponding training change categories for the previous changes.

[0069] Example 13. The method according to Example 12 further includes: for each of the previous manufacturing change specifications, computing one or more key phrases associated with the previous manufacturing change specification at a natural language processing model, wherein the first training data also includes the key phrases.

[0070] Example 14. The method according to Example 12 or 13 further includes training the second trained machine learning model using second training data, the second training data including: the plurality of previous manufacturing change specifications; the plurality of training change categories of the previous changes indicated in the previous manufacturing change specifications; and corresponding plurality of previous change magnitude classifications associated with the previous changes.

[0071] Example 15. The method according to any one of Examples 11-14, wherein the change magnitude classification is a multidimensional binary classification.

[0072] Example 16. The method according to any one of Examples 11-15 further includes: after outputting the one or more predicted change categories and the change magnitude classifications, receiving one or more ground reality change categories and ground reality change magnitude classifications via the user interface; and performing additional training on the first trained machine learning model and the second trained machine learning model using additional training data including the one or more ground reality change categories and the ground reality change magnitude classifications.

[0073] Example 17. The method according to any one of Examples 11-16, wherein the one or more predicted change categories specify one or more manufacturing devices included in the manufacturing system.

[0074] Example 18. The method according to any one of Examples 11-17, wherein the one or more predicted change categories specify one or more manufacturing steps of the manufacturing process.

[0075] Example 19. The method according to any one of Examples 11-18, wherein: the manufacturing system is configured to manufacture an aircraft; and the one or more predicted change categories specify one or more components of the aircraft manufactured at the manufacturing system.

[0076] Example 20. A computing system comprising: one or more processing devices configured to: train a first trained machine learning model using first training data, the first training data including: a plurality of previous manufacturing change specifications indicating corresponding previous changes to a manufacturing system; a plurality of corresponding training change categories of the previous changes; and a plurality of key phrases, each associated with one of the previous manufacturing change specifications; train a second trained machine learning model using second training data, the second training data including: the plurality of previous manufacturing change specifications; the plurality of training change categories of the previous changes indicated in the previous manufacturing change specifications; a plurality of corresponding previous change magnitude classifications associated with the previous changes; and the plurality of key phrases; receive a manufacturing change specification for a proposed change to the manufacturing system; at the first trained machine learning model, compute one or more predicted change categories of the proposed change; at the second trained machine learning model, compute, at least in part, a change magnitude classification associated with the proposed change based on the one or more predicted change categories; and output the one or more predicted change categories and the change magnitude classifications to a user interface.

[0077] As used herein, “and / or” is limited to inclusion or V, as specified in the following truth table: It should be understood that the configurations and / or methods described herein are exemplary in nature, and these specific embodiments or examples should not be considered limiting, as many variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. Thus, the various actions shown and / or described may be performed in the order shown and / or described, in a different order, in parallel, or omitted. Similarly, the order of the above processes may be changed.

[0078] The subject matter of this disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations disclosed herein, as well as any and all equivalents thereof.

[0079] Parts list Computing System 10 Processing equipment 12 Memory device 14 Input device 16 Output device 18 Manufacturing Change Specification 20 Proposed Change 22 Manufacturing System 24 Manufacturing equipment 25 Manufacturing process 26 Manufacturing step 27 Aircraft 28 Component 29 The first trained machine learning model 30 Predicted change category 32 The second trained machine learning model 40 Change range classification 42 User Interface 50 First training data 60 Previous manufacturing change specification 62 Training Change Category 64 Key phrase 66 Natural Language Processing Model 68 First Machine Learning Model 70 Change category 72 in training time prediction First loss function 74 Model Update 76 Training iterations 78 Second training data 80 Previous change range classification 82 Second machine learning model 90 Classification of Training Time Changes (92) Second loss function 94 Model Update 96 Training iterations 98 Table 100 Table 102, Part 2 Ground Status Change Category 110 Ground Condition Change Classification 112 Additional training data 114 The first machine learning model, after further training, 116 The second machine learning model 118 was further trained. Method 200 Step 202 Step 204 Step 206 Step 208 Step 210 Step 212 Step 214 Step 216 Step 218 Computing System 300 Processing circuit system 302 volatile memory 304 Non-volatile storage device 306 Display Subsystem 308 Input Subsystem 310 Communication Subsystem 312

Claims

1. A computing system (10), comprising: One or more processing devices (12) are configured to: Receive manufacturing change specification (20) for proposed changes (22) to manufacturing system (24); At the first trained machine learning model (30), one or more predicted change categories (32) of the proposed change (22) are calculated. At the second trained machine learning model (40), a change magnitude classification (42) associated with the proposed change (22) is calculated, wherein the second trained machine learning model (40) is configured to calculate the change magnitude classification (42) at least in part based on the one or more predicted change categories (32). as well as The one or more predicted change categories (32) and the change magnitude categories (42) are output to the user interface (50).

2. The computing system (10) of claim 1, wherein the one or more processing devices (12) are further configured to train the first trained machine learning model (30) using first training data (60), the first training data (60) comprising: Multiple prior manufacturing change specifications (62) indicate corresponding prior changes to the manufacturing system (24); as well as The corresponding multiple training change categories (64) of the previous changes.

3. The computing system (10) according to claim 2, wherein: For each of the aforementioned prior manufacturing change specifications (62), the one or more processing devices (12) are further configured to compute one or more key phrases (66) associated with the prior manufacturing change specification (62) at a natural language processing model (68); and The first training data (60) also includes the key phrase (66).

4. The computing system (10) of claim 2, wherein the one or more processing devices (12) are further configured to train the second trained machine learning model (40) using second training data (80), the second training data (80) comprising: The aforementioned multiple previous manufacturing change specifications (62); The plurality of training change categories (64) of the previous changes indicated in the previous manufacturing change specification (62); as well as The corresponding multiple previous change magnitude classifications associated with the aforementioned previous changes (82).

5. The computing system (10) according to claim 1, wherein the change category classification (42) is a multidimensional binary classification.

6. The computing system (10) according to claim 1, wherein the one or more processing devices (12) are further configured to: After outputting the one or more predicted change categories (32) and the change magnitude classification (42), one or more ground condition change categories (110) and ground condition change magnitude classifications (112) are received via the user interface (50); and Additional training is performed on the first trained machine learning model (30) and the second trained machine learning model (40) using additional training data including the one or more ground reality change categories (110) and the ground reality change magnitude classification (112).

7. The computing system (10) of claim 1, wherein the one or more predicted change categories (32) specify one or more manufacturing devices (25) included in the manufacturing system (24).

8. The computing system (10) according to claim 1, wherein the one or more predicted change categories (32) specify one or more manufacturing steps (27) of the manufacturing process (26).

9. The computing system (10) according to claim 1, wherein: The manufacturing system (24) is configured to manufacture aircraft (28); and The one or more predicted change categories (32) specify one or more components (29) of the aircraft (28) manufactured at the manufacturing system (24).

10. The computing system (10) of claim 1, wherein the first trained machine learning model (30) outputs an n-dimensional vector of Boolean values, where n is the number of potential change categories.

Images