System and method for assembly line suitability analysis

Abstract

The invention provides a system and method for assembly line suitability analysis. A system and method includes acquiring data related to operation of an assembly line and generating a virtual environment of the assembly line using the acquired data. A virtual environment is defined using an assembly process model synchronized with real-time assembly line data. Real-time assembly line data, historical data, and sensor data are used to simulate tasks of one or more production line workers. The assembly line is rebalanced in real-time based on the simulation to reduce the travel time for one or more production line workers to perform tasks.

Description

Technical Field

[0001] The present disclosure relates to monitoring systems. More particularly, the present disclosure relates to systems and methods for operating monitoring systems incorporating digital twins. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0003] The manufacturing process can be supervised by a vision-based system to monitor the process. The monitoring system can be implemented by using a variety of devices, systems and processes. Some monitoring systems include imaging devices (such as video cameras and infrared cameras) for various purposes (such as for monitoring production performance). The digitization of processes such as assembly lines can also be used as part of monitoring. However, performing the digitization of assembly lines can be challenging because it can be difficult to obtain the latest information without interfering with operations along the assembly line (such as workers working at different workstations along the assembly line). Real-time monitoring can also be difficult to perform because such monitoring combines data sources from different operational databases with a large amount of data to be filtered.

[0004] The present disclosure addresses these and other problems associated with using digital systems to monitor production processes. Summary of the Invention

[0005] This section provides a general summary of the disclosure and is not a comprehensive disclosure of its full scope or all of its features.

[0006] The present disclosure provides a computerized method, which includes: acquiring data related to the operation of an assembly line; using the acquired data to generate a virtual environment of the assembly line, wherein the virtual environment is defined using an assembly process model synchronized with real-time assembly line data; using real-time assembly line data, historical data and sensor data to simulate the tasks of one or more production line workers; and rebalancing the assembly line in real time based on the simulation to reduce the walking time of one or more production line workers to perform tasks; wherein the simulation data includes using data acquired from one or more sensors along the assembly line to approximate the walking time of a set of tasks; wherein the one or more sensors include imaging devices, the imaging devices including cameras, laser sensors or infrared sensors; wherein the virtual environment includes a digital twin, and the acquired data includes data from one or more databases, the data describing the assembly process of the assembly line, vehicles to be assembled along the assembly line, the production line layout of the assembly line, the location of parts, or a combination thereof; wherein the one or more databases include a production line layout database, an assembly process database, a parts database, a factory IoT database, a vehicle CAD database and a manufacturing CAD library database; wherein IoT messages are used to transfer the assembly line to the assembly line. wherein the assembly line comprises a plurality of cells, and the simulation comprises simulating different cell layouts of one or more of the plurality of cells and performing a sensitivity analysis to identify a cell layout optimization corresponding to minimized travel time; the computerized method further comprises using an optimization algorithm to rebalance the assembly line, wherein at least a portion of the assembly line is rebalanced using one or more precedence constraints for new parts to be assembled along the assembly line or new positions of one or more items within one or more cells; the computerized method further comprises displaying a virtual environment of the assembly line, the virtual environment having a plurality of cells, the plurality of cells comprising a plurality of items represented as a plurality of boxes within the plurality of cells and a plurality of travel patterns represented by a plurality of lines, and in response to receiving user input, virtually moving one or more of the boxes, updating the one or more travel patterns, including moving one or more of the plurality of lines, thereby showing the updated travel patterns; and the computerized method further comprises using one or more APIs to cause the virtual environment to access one or more optimization functions to visualize optimal solutions for the one or more updated travel patterns corresponding to the one or more different vehicle mixes along the assembly line.

[0007] The present disclosure provides a system comprising: a plurality of sensors configured to acquire movement data associated with movement of workers at an assembly line; and an analysis system receiving the movement data and configured to: acquire operation data associated with operation of the assembly line; generate a virtual environment of the assembly line using the acquired movement data and operation data, wherein the virtual environment is defined using an assembly process model synchronized with real-time assembly line data; simulate tasks of one or more production line workers using real-time assembly line data, historical data, and the acquired movement data; and rebalance the assembly line in real time based on the simulation to reduce the walking time of one or more production line workers performing tasks; wherein the analysis system is further configured to simulate the data by approximating the walking time of a set of tasks using the acquired movement data; wherein the plurality of sensors include imaging devices including cameras, laser sensors, or infrared sensors; wherein the virtual environment includes a digital twin, and the acquired data includes data from one or more databases describing an assembly process of the assembly line, vehicles to be assembled along the assembly line, a production line layout of the assembly line, a part location, or a combination thereof; wherein the one or more databases include a production line layout database, an assembly process database, a part database, a factory IoT database, a vehicle C AD database and manufacturing CAD library database; wherein IoT messages are used to synchronize the assembly process model with real-time assembly line data; wherein the assembly line includes a plurality of cells, and the analysis system is further configured to simulate different cell layouts of one or more cells in the plurality of cells, perform sensitivity analysis to identify the cell layout optimization corresponding to minimized walking time and display a virtual environment of the assembly line, the virtual environment having a plurality of cells, the plurality of cells including a plurality of items represented as a plurality of boxes within the plurality of cells and a plurality of walking patterns represented by a plurality of lines, and in response to receiving user input, virtually move one or more of the boxes, update the one or more walking patterns, including moving and moving one or more of the plurality of lines to show an updated walking pattern; wherein the analysis system is further configured to use the one or more APIs to cause the virtual environment to access the one or more optimization functions to visualize the one or more updated optimal solutions for the walking pattern corresponding to the one or more different vehicle mixes along the assembly line, and wherein the assembly line includes a plurality of cells, wherein the analysis system is further configured to use the optimization algorithm to rebalance the assembly line, and wherein at least a portion of the assembly line is rebalanced using one or more precedence constraints for new parts to be assembled along the assembly line or new positions of one or more items within one or more of the plurality of cells.

[0008] The present disclosure provides one or more non-transitory computer-readable media storing processor-executable instructions that, when executed by at least one processor, cause the at least one processor to: acquire data related to the operation of an assembly line; use the acquired data to generate a virtual environment for the assembly line, wherein the virtual environment is defined using an assembly process model that is synchronized with real-time assembly line data; use real-time assembly line data, historical data, and sensor data to simulate tasks of one or more production line workers; and rebalance the assembly line in real time based on the simulation to reduce walking time for one or more production line workers to perform tasks.

[0009] Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order that the present disclosure may be better understood, various forms of the present disclosure will now be described by way of example with reference to the accompanying drawings, in which:

[0011] Figure 1 An overall system for monitoring a production process according to various embodiments is shown;

[0012] Figure 2 A system for digital twin assembly line suitability analysis according to various embodiments is shown;

[0013] Figure 3 shows representations of different production lines of an assembly line using diagrams according to various embodiments;

[0014] Figure 4 shows a virtual representation of a unit showing a walking pattern according to various embodiments;

[0015] Figure 5 shows a prioritization diagram according to various embodiments;

[0016] Figure 6 is an example of a historical build sequence of a part according to various embodiments;

[0017] Figure 7 According to various embodiments, Figure 6 Priority graph generated from the historical build sequence of the parts in ;

[0018] Figure 8 illustrates a user interface according to various embodiments;

[0019] Figure 9 Another user interface according to various embodiments is shown;

[0020] Figure 10 shows simulations according to various embodiments; and

[0021] Figure 11 is a flow chart illustrating an example method for performing assembly line suitability analysis using a digital twin, according to various implementations.

[0022] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way. DETAILED DESCRIPTION

[0023] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses. It should be understood that throughout the drawings, corresponding reference numerals indicate like or corresponding parts and features.

[0024] The present disclosure provides a means for monitoring a production process (such as an automotive process) using digitization (e.g., a digital twin) in various examples. For example, in various examples, a digital twin of the final assembly of an automobile connects products, parts, tools, processes, and / or personnel, which allows monitoring of production performance and enables assembly line suitability analysis (ALFA), which quantifies the impact of decisions in real time and allows identification of opportunities for assembly line optimization. In some examples, digitization combined with real-time monitoring and optimization allows decisions to be made to optimize operations at one or more workstations along the assembly line (such as at the final assembly workstation). In some examples, decision making allows for more efficient and productive assembly line planning.

[0025] One or more embodiments described herein can reduce or eliminate the challenging and complex processes of assembly line monitoring, such as final assembly line monitoring with respect to digitization, real-time monitoring and optimization, and decision making. For example, various embodiments reduce or eliminate the digitization challenges of a digital twin that requires part and tool locations, worker walking patterns, and secure access to up-to-date information from an operations database while respecting worker privacy and not interfering with operations (e.g., allowing for rapid querying of an operations database to obtain large data sets). Various embodiments reduce or eliminate the real-time monitoring challenges of combining several data sources from different operations databases and efficiently displaying the most relevant information to a user. For example, monitoring is focused on relevant information without having to apply filters or navigate a complex user interface, such as when a user is on the floor, holding a tablet computer while walking around, or when a supervisor is monitoring an entire assembly line (also referred to as a production line).

[0026] Some examples provide digital twin implementations configured to perform assembly line suitability analysis (ALFA). For example, ALFA is provided in conjunction with automotive assembly line planning, where one or more implementations include a digital twin that drives ALFA. Figure 1As shown, an example shows a schematic block diagram of a monitoring system 100 such as one for monitoring a final assembly line process. The final assembly line typically has a logical hierarchy provided as follows: a factory (or a separate section of a factory) contains production lines. A production line or a group of production lines focuses on a specific domain, such as a chassis or body assembly. Each production line consists of cells (also called workstations), which can be logical or physical in nature. Operators (or workers) are assigned to specific cells. Operations (or work instructions, work elements) are assigned to workers in the cells. It should be understood that there is a balance between the speed of the production line, the complexity of the operation, the number of tasks assigned to the operator, and running the production line smoothly without stopping, among other factors. An operator taking more time than expected to perform an operation may result in a temporary production line stop immediately, or later when enough "extra time" has been consumed in performing the operation.

[0027] It is desirable to run a production line as quickly as possible with the fewest number of units possible without causing any manufacturing or assembly issues, while adhering to factory requirements for ergonomics, proper operation, and the like. Some examples use monitoring systems to perform line balancing optimization operations on one or more assembly lines. Furthermore, in some examples, cell layout optimization is performed to organize the layout of cells to minimize non-value-added steps, such as travel time. In conjunction with line balancing and / or cell layout optimization, real-time monitoring of the production line identifies potential issues and optimization opportunities.

[0028] like Figure 1 As can be seen in FIG, the monitoring system includes a digital twin 102, which can be displayed on a screen 104 of a computing device 106 (such as a computer or other processing machine). The computing device 106 can be coupled to one or more sensors 110, such as an imaging device 100, via a network 108, which is configured to capture one or more images or other sensor data of one or more units 112 (e.g., final assembly workstations). For example, in some embodiments, the one or more sensors 110 are configured to acquire sensor data associated with one or more workers 114 (e.g., operators), which sensor data allows for approximate calculation of walk times for a set of tasks for the one or more workers 114 as described in more detail herein and can be used, for example, to rebalance one or more production lines. It should be understood that the one or more sensors 110 can be any type of sensor that captures different types of sensed data in any type of monitored area (such as along an assembly line or portion thereof).

[0029] The network 108 may include any one or a combination of various networks, such as a data network, a telephone network, a cellular network, a cable network, a wireless network, a private network, a public network, a local area network (LAN), a wide area network (WAN), and the Internet. In some cases, the network 108 may support communication technologies such as Bluetooth, cellular, near field communication (NFC), Wi-Fi, and / or Wi-Fi Direct. In some examples, the computing device 106 may be directly coupled to one or more sensors 110 via a cable or fiber optic cable, and communicatively coupled to other components such as a server system 116 via the network 108.

[0030] The one or more sensors 110 may include various types of imaging devices, such as, for example, a digital camera configured to take snapshots of the one or more units 112 (periodically, intermittently, or as needed), a video camera configured to generate video files based on video monitoring of the one or more units 112, and / or an infrared camera configured to generate infrared images (snapshots and / or videos) of the one or more units 112, among others. The one or more sensors 110 may also include other components or sensing devices, such as motion detectors or speed sensors. Motion detectors may be used in some applications to trigger image capture (or video capture) in the one or more sensors 110. Speed ​​detectors may be used to obtain motion information associated with one or more moving objects in the one or more units 112, such as the walking speed of one or more workers 114. In some embodiments, the motion information may be a digital speed value for the moving objects in the one or more units 112. The digital speed value may be transmitted to the computing device 106 or server system 116 via the network 108. The digital speed value may also include a timestamp indicating the time when the speed of the moving object was captured. In some implementations, computing device 106 may determine the speed of an object (e.g., walking speed, pace, number of steps, etc.) by processing multiple snapshots or video frames captured by one or more sensors 110 and transmitted to computing device 106 or server system 116.

[0031] It should be noted that one or more cells 112 may include, for example, different monitored areas along an assembly line. One or more cells 112 may contain parts and equipment useful for assembling different vehicle types, such as vehicles with internal combustion engines (ICEs) and hybrid vehicles. However, a monitored area, such as one or more cells 112, may be any of a variety of other types of areas placed under monitoring, such as different workstations along one or more assembly lines used to assemble one or more different types of vehicles. Each of these areas may include a combination of moving objects (e.g., vehicles being assembled and workers 114) and stationary objects (e.g., equipment, parts, tools, storage containers, machines, packaging, columns, racks, etc.) during one or more time periods. Thus, when the monitored area includes one or more cells 112 located on a factory floor used to assemble vehicles, in some examples, the moving objects and stationary objects are any objects involved in the assembly process.

[0032] In some examples, when two different vehicle types are moving along an assembly line, each vehicle may have different unique operations to be performed in conjunction with the assembly of each vehicle. For example, consider a case where vehicle A is an ICE engine with a turbocharger and vehicle B is a hybrid vehicle. Each vehicle type may have some unique operations, and some common operations may require different times to complete (e.g., the battery partially hinders installation on an HEV vehicle). Let t c is the takt time, which specifies the global maximum time for the vehicle to work on each unit 112. Let t i w is the average amount of time that a worker 114 in a cell 112i spends working instead of waiting for a new vehicle. Given N cells in a production line, let T = {t i w :1<=i<=N}. Let t wavg is the average value of the elements of T. Assume

[0033] To prevent overworked units 112 from skewing the average, the line utilization can be defined as:

[0034]

[0035] A balanced line means that each worker 114 in a cell 112 works the same amount of time. In other words, if d = sup T - inf T, then ideally, d = 0. An efficient line can then be defined as a balanced line where the utilization is greater than a defined amount. Note that u <= 1, since t wavg ≤t cThus, various examples perform line balancing to minimize d and 1-u, as described in more detail herein.

[0036] Sensor data (e.g., images captured by one or more sensors 110) can be received by computing device 106, processed to obtain information about one or more cells 112, and used to generate digital twin 102 displayed on screen 104. In at least some embodiments where one or more sensors 110 are configured to transmit real-time images of a monitored area within one or more cells 112, digital twin 102 can be updated in real time to render a temporal digital replica of one or more cells 112 and used for ALFA, as described in more detail herein. When rendered in real time, any changes to the monitored area (such as, for example, personnel 114 or vehicles entering or exiting the monitored area) can be detected by computing device 106 and used for various purposes, including one or more processes or operations described herein.

[0037] In one exemplary application, the computing device 106 may be configured to use the acquired data to generate a virtual environment of an assembly line, wherein the virtual environment is defined using an assembly process model that is synchronized with the real-time assembly line data, as described in more detail herein. The computing device 106 may also be configured to simulate tasks of one or more of the workers 114 (e.g., production line workers) using the real-time assembly line data, historical data, and sensor data from one or more sensors 110 to allow the assembly line to be rebalanced in real time based on the simulation to reduce the walking time of one or more workers 114 to perform one or more tasks. It should be noted that the computing device 106 may operate in conjunction with the server system 116 to perform one or more processes or operations as described herein. For example, the server system 116 may be a cloud-based system comprising one or more computing devices that include components such as processors and memory devices. In Figure 1 In the illustrative example shown, server system 116 may include at least one computer having at least a processor and memory. Memory, an example of a non-transitory computer-readable medium, may be used to store various types of information, such as real-time assembly line data, historical data, and sensor data. Server system 116 may respond to requests received from computers, such as computing device 106, by transmitting the requested information via network 108.

[0038] The information received from the server system 116 can be used by the computing device 106 to process sensor data (e.g., images) and other data received from one or more sensors 110 and operate the digital twin 104 according to one or more embodiments of the present disclosure. Methods for processing sensor data and / or operating the digital twin 104 can provide various advantages, such as optimizing the balance of the assembly line to operate in various environments and various applications, and using digitization and real-time monitoring to assemble different types of vehicles along the assembly line, as described in more detail herein. It should be noted that the various components described herein can cooperate to acquire data and generate desired information about the monitored area.

[0039] For example, one or more embodiments allow for the optimization of the layout of workstations to facilitate a balanced assembly line operation. Specifically, the efficiency of the above-described line u is defined across the entire line. It should be understood that in various examples, "busy" does not mean "value added." For example, "non-value added" time is due to walking from the mounting point on the vehicle to the rack to grab the part and to the tool, bin, etc. Some of these walking patterns are desirable but can be minimized by optimizing the location of the racks, bins, and parts in the cell using one or more embodiments. For example, let t 总 is the amount of time that worker 114 spends not waiting for a vehicle to arrive. Let t 增值 The time it takes to install the part in the vehicle (excluding walking, breaking boxes, moving items, preparing tools, etc.). Then, the efficiency of unit 112 (e c ) can be defined as:

[0040]

[0041] Thus, in some examples, workstation layout optimization in one or more units 112 is performed to identify c The appropriate layout of items in the workstations is maximized while adhering to design constraints such as those imposed by different requirements. For example, in the case of major and minor model changes, different parts can be used, and the operation time within one or more cells 112 based on the cell layout can be changed. In one or more examples, the positioning of items within one or more cells 112 and the travel paths of workers 114 can be changed in response to changes in vehicle models or other operational changes to rebalance the production line, which can occur in real time (e.g., by simulating tasks using the digital twin 104).

[0042] As an example, if the order of vehicle types changes, the corresponding operation time will change. For example, determining the vehicle sequence of (HEV, HEV, fuel) may be more complex than the vehicle sequence of (HEV, fuel, HEV). This is because the HEV type may cause the operator 114 to work for more than tc , but the operator 114 can recover lost time when working on gasoline vehicle types. The input also uses multiple heterogeneous data sources, such as CAD layouts, IoT messages, and relational databases, and there are additional constraints to consider, such as tool constraints; constraints from "lessons learned", ergonomics, operator skills, lighting and visibility; and so on. In addition, by retrieving information in real time to use the latest information (for example, in the case where an industrial engineer moves a rack on the site to a different location), one or more examples allow for corresponding updates of the digital twin 104. That is, in various examples, the digital twin 104 is updated more efficiently to allow for more efficient assembly line operations (for example, more efficient rebalancing and unit layout).

[0043] In some examples, the digital twin 104 provides an efficient and effective method for performing line balancing and cell layout to facilitate other assembly line fitness analysis tasks. In some examples, fitness analysis is the process of measuring the efficiency of different manufacturing domains in near real time, predicting future scenarios and problems, and identifying possible solutions to identified problems. In one example, the assembly line fitness analysis system has the following properties:

[0044] 1. Descriptive: Provides a complete representation of site operations and allows users to “slice and dice” data to quickly gain insights. Combines data from various manufacturing systems for real-time analysis to help modify assembly work instructions to improve production line efficiency.

[0045] 2. Predictive: Highlight existing problems and predict future issues before they occur. See the effects of decisions to be made.

[0046] 3. Prescriptive: Optimizes cell and line layouts and allows creation of “what-if” scenarios. Automatically generates sequencing constraints by taking user decisions into account.

[0047] Figure 2 An example of a system 200 for digital twin ALFA is shown in . The illustrated system 200 integrates multiple data sources (e.g., heterogeneous data sources or databases), such as data from an assembly process database 202, a vehicle CAD database 204, a production line layout database 206, a parts database 208 (e.g., a parts location database), a factory IoT database 210, and a manufacturing CAD library database 212 to generate an assembly process model 214, while synchronizing the assembly process model 214 with the actual assembly line through IoT messaging. Thus, near real-time visibility is provided to assembly line operations. In some examples, the system 200 integrates historical data, current data, and real-time IoT messages to identify opportunities for improving efficiency, as described in more detail herein.

[0048] In some examples, the assembly process described herein involves steps (e.g., different tasks) performed at each cell 112 to assemble a vehicle. The steps are assigned a Modular Arrangement of Predetermined Time Standards (MODAPTS) code in various examples. Each step is associated with a part and / or tool, where applicable. For example, the step "When the bin is empty, break it and throw it" does not have any parts associated with it. Details of these parts and tools, such as size, description, and the vehicle configuration for which the part / tool ​​is applicable, are obtained from the parts database 208. Each step has an associated time, obtained through MODAPTS time estimates or actual time studies at the site. The factory IoT database 210 provides information such as the actual sequence of vehicle types moving across the site, messages indicating which parts have been scanned, messages indicating which cell 112 caused the line to stop and for what reason (e.g., missed scan, manual stop, etc.), and, in some cases, messages indicating which racks have been refilled with parts. The vehicle CAD database 204 contains data related to the mounting points of parts in the vehicle. Since this is a single point, it is typically the center of mass of the part and is therefore an approximation in various examples. The vehicle CAD database 204 also includes information such as part weight and, in some cases, for large parts, a three-dimensional (3D) representation of the part that can be integrated into a 3D environment. The manufacturing CAD library database 212 contains 3D representations of tools, racks, boxes, etc.

[0049] In one or more examples, an application programming interface (API) layer allows various optimization algorithms and services to be integrated within the digital twin 104. For example, ergonomic rules may change frequently based on new parts. Some rules must be implemented quickly when updated. In addition, there may be special business rules such as: "If two different parts fit on the same tool, they must be in different cells." In some examples, such rules and constraint checks, which typically respond with a yes / no response, are suitable for implementation as an API layer with one or more APIs 220 (e.g., optimization and analytics service APIs). This is because different factories may have slightly different rules, and in one or more examples, these rules are managed outside the digital twin 104. The API layer also allows the digital twin 102 to synchronize virtual objects with the real world (such as the location of workers 114 and the location of parts, racks, and boxes in each cell 112).

[0050] It may be difficult to keep the layout information of the cells 112 up to date because industrial engineers may find a good optimal solution online when observing a particular problem. Therefore, one or more examples include a vision system (e.g., one or more sensors 110 providing visual data) that monitors each cell 112 and extracts layout information. In addition, the vision system can monitor when workers 114 enter and leave the cells 112. This information is useful in the calculation of walking pattern analysis, as described in more detail herein. For example, the time it takes for a worker 114 (e.g., an operator) to walk to a vehicle and perform a set of operations and then return can be accurately calculated. In some examples, the digital twin 102 integrates digital versions of lean manufacturing tools, such as a virtual whiteboard 216, charts (such as a Yamazumi board 218), etc., and allows different users to interact with these tools, in some examples, simultaneously. In some examples, the digital twin 102 also provides simulations or predictions.

[0051] Virtual changes can be made within the digital twin 102 to see how the system changes before any changes are made to the original system. To achieve this, vision-based and sensor-based synchronization are used to maintain some of the information required by the digital twin 102, such as rack position identification, vehicle position identification, worker position identification (visually or through synchronization scanning messages), etc. In the absence of this information, one or more examples utilize estimation algorithms to approximate the unknown state. For example, if the production line speed is known, the position of the vehicles on the production line is obtained from the sensor 110, and the vision-based trigger only indicates when the worker 114 has entered or left the cell 112, one or more embodiments estimate the position of the operator outside the cell 112 while working on the vehicle, and then correct the "history" when the worker 114 walks back into the cell 112. In some examples, this type of estimation is performed as a service and integrated via the API 220 to simplify the digital twin 102 in some examples. Therefore, coordinating and combining all of this information allows for optimization of the production line and cell layout to be considered.

[0052] In some examples, a processing engine (e.g., a game engine) is used to visualize the assembly process and provide interactive tools to support human decision makers. Visualizations are built for 3D cell layout representations, 3D production line representations, and 2D digital versions of lean manufacturing tools, such as Figure 3 and Figure 4 For example, Figure 3 An updated Yamazumi board 218 is shown showing parts 230, tools 232, operations 234, and different mixes 236 on the production line, among other data. Figure 4A digital virtualization 300 of a virtual environment generated according to an embodiment is shown. The digital virtualization 300 shows a walking pattern 302 of a worker 114 in a cell 112, wherein in various examples, walking pattern analysis and cell layout optimization can be performed in isolation from the rest of the production line. In some examples, the digital virtualization 300 is a representation of a digital twin 102 (which can be generated with a CAD tool in some other examples) compiled on various different platforms (such as MAC, PC, Android tablet, Apple tablet, etc.), with virtual reality and augmented reality integrated, supporting multiple drivers and GPU features. In some examples, when multiple users are optimizing a production line at the same time, a "multi-person" style (and "freestyle" method) interactivity of the digital virtualization 300 is provided, wherein network synchronization is performed by a processing engine. It should be noted that a bar chart of the source of the cycle time in any display process that graphically represents a process can be used.

[0053] In various examples, the digital twin 102 is configured to provide insights, visualizations, simulations, analyses, and / or predictions while supporting multiple entities (e.g., industrial engineers running a production line, production line supervisors, engineers working on a new production line, etc.). In some examples, a user interface (UI) is provided that has separate and dedicated views for performing different tasks related to production line balancing, cell layout optimization, etc., rather than a single UI. However, in some examples, a single UI may be provided.

[0054] In various examples, cell layout and worker (e.g., operator) travel pattern analysis is performed. For example, cell layout optimization is performed to minimize the duration of non-value-added tasks by reducing travel time. In some examples, this results in a faster production line or a reduced number of cells. Figure 4 , and as a box 304 (e.g., a digital box representation of a bin, rack, part (inventory), etc.) is moved (e.g., virtually dragged) around the unit 112, the walk pattern 302 is updated while the box 304 is moving. In some examples, the system 200 interacts with the data and performs sensitivity analysis, as described in more detail herein, to generate an updated walk pattern 302 for display. For example, Figure 3The operations shown in the Yamazumi board 218 shown in can be imported into the cell 112, which then allows the user to analyze how different variations of the cell layout affect worker performance. A series of different vehicle mixes can be provided to analyze the walking patterns 302 for that particular sequence. In operation, using the API 220, the digital twin 102 can call the optimization function and visualize the proposed optimal solution and, if necessary, manually adjust the proposed solution, as described in more detail herein. It should be noted that if more than one worker 114 is working in the cell 112, the analysis becomes more complex (e.g., it may be difficult to calculate the actual walking path due to the possibility of interference). In this case, the processing engine uses path finding capabilities to assist in the simulation. For example, each agent in the simulation can have a corresponding script that implements the associated behavioral logic, and when the simulation is run, the actual walking path can be obtained based on how the agents interact.

[0055] In some examples, monitoring activity on the assembly line is combined with optimization. As discussed herein, in one or more examples, the digital twin 102 calls an API 220 (e.g., a vision system API 220) to obtain information about when the worker 114 has left and entered the cell 112 and where the worker 112 has walked within the cell. The digital twin 102 is configured to track the actual visible walking path. In some examples, the remainder of the path is estimated, such as by using IoT messages for scanning, tool actuation, and positioning of the vehicle on the production line as described herein. It is then possible to go back in time to understand the reason for the production line stoppage. In some examples, the digital twin 102 aggregates data related to the production line stoppage and generates an error message in the digital twin 102. Figure 3 and Figure 4 The "hot zone" is highlighted on the part of the screen.

[0056] In some examples, the digital twin 102 also allows for analysis of "what-if" scenarios. For example, production line optimization that extends beyond cell layout requires understanding the prioritization constraints on the vehicles being built. In some examples, these prioritization constraints can be captured in the form of a directed acyclic graph (DAG) (also referred to as a prioritization constraint graph 400), a portion of which is shown in FIG. Figure 5, which has one or more rules. For example, the rule includes that the root of the precedence constraint graph 400 is a start node, and the start node can be artificially added to identify the beginning of the precedence constraint graph 400. Each node on the precedence constraint graph 400 has a path to an end node, and the end node can be artificially added at the end of the precedence constraint graph 400. In addition, the precedence constraint graph 400 has the following no shortcut property: for any two nodes A and B on the DAG, if node A is directly before node B, that is, there is an arrow connecting A directly to B, then there is no other path from A to B except this direct connection. That is, this directional connection is not a shortcut to a longer path.

[0057] In some examples, the precedence constraint graph 400 is used as follows: for any arbitrary node n on the precedence constraint graph 400, all previous nodes that have a path to n must be installed. It will be understood that there can be a maximum graph, a minimum graph, and a target graph that lies in between, and algorithms are provided for mining the graph. In some examples, it is also considered that the nodes on the precedence constraint graph 400 are parts. For example, parts can change between model years, and parts for the same model year and vehicle type can change depending on the build options and trim levels selected (e.g., HEV, etc.). This means that some parts are not always present on the precedence constraint graph 400, and therefore for each particular build, nodes on the precedence constraint graph 400 that are not applicable are short-circuited in some examples (connecting the previous node to its successor node).

[0058] exist Figure 5 In the example shown in FIG, the nodes correspond to part IDs and installation sequences. In this example, the part with part ID = 0 must be installed first. Then, the part with part ID = 1 or part ID = 2 can be installed. Using the various embodiments described herein and applied to part ID 10, in order to install the part corresponding to or associated with part ID 10, part ID 8 must be installed, and before part ID 8 is installed, part ID 1 must be installed. As another example and considering part ID 20, before installing that part, part ID 6, part ID 11, and part ID 5 must be installed. Thus, the nodes define the following installation sequence:

[0059] Before installing part ID 11, you must install part ID 10.

[0060] Before installing part ID 10, you must install part ID 8.

[0061] Before installing part ID 8, you must install part ID 1.

[0062] Before installing part ID 1, you must install part ID 0.

[0063] Before installing part ID 6, you must install part ID 1.

[0064] Before installing part ID 1, you must install part ID 0.

[0065] Before installing part ID 5, part ID 1 must be installed, and before installing part ID 1, part ID 0 must be installed.

[0066] Therefore, it should be understood that each node on the diagram 400 must be installed before the subsequent node. That is, a part cannot be installed until the part before it is installed. The diagram 400 represents the installation order of the parts using the part IDs corresponding to the various nodes and is used in one or more examples as described in more detail herein. It should be noted that if a part is not required for the vehicle (e.g., an optional part), the node is "short-circuited" so that the output of the node is connected to the input node.

[0067] In some examples, the interpretation of historical data can vary between domains. For example, if a log file L has the following entries: L1: {A, B, C, D}, L2: {A, B, D}, and L3: {A, B, C}, the following decision is made: choose "whenever C and D appear, C must come before D" or "C and D are independent of each other." In these examples, a heuristic algorithm is used to consider "optional" parts, which is generated using set theory. The algorithm constructs the precedence constraint graph 400 as it scans the historical data. Therefore, if the log is sorted so that the newer information is first, the newer information will be prioritized when constructing the precedence constraint graph 400. Figure 5 A portion of a generated precedence constraint graph 400 is shown. The precedence constraint graph 400 represents a large number of parts and is used by one or more processing engines in various examples because the precedence constraint graph 400 is complex to understand visually. That is, in some examples, the precedence constraint graph 400 cannot be understood visually.

[0068] In some examples, the algorithm uses Figure 6The algorithm takes as input a historical build sequence 500 of the part shown, where each row 502 represents a historical valid sequence. The algorithm processes the data in the historical build sequence 500 of the part column by column starting from the top column 502 to the last column in various examples, and builds the precedence constraint graph 400 as the algorithm receives and processes the data (e.g., dynamically generates the precedence constraint graph 400). In some examples, older sequences are positioned lower than newer sequences. Each part number in the sequence appears only once. If the same part appears multiple times, a number can be appended to the end of the part number to make each part unique. Without loss of generality, assume that the algorithm completes in the third column 504 (column 3) and therefore a partial precedence constraint graph 400 already exists, and the algorithm now starts at column 506 (column 4). In one example, the algorithm performs the following operations:

[0069] 1. Loop through all r rows (c, then e) of column 506 (column 4). Each row represents a marked node n 4 i , where 1<=i<=r.

[0070] 2. If node n 4 i If it is already in the diagram, do nothing and go to step 1.

[0071] 3. If there are no more rows 502, move to the next column 508.

[0072] 4. If there are no more columns, write the precedence constraint graph 400 and terminate the algorithm.

[0073] 5. For each n 4 i , find the table with the same 4 i All nodes with the same label. In the case of node c, it is unique. In the case of e, there are two e nodes in the table. Define I as: With n 4 i Same label.

[0074] 6. For each n∈I, get all ordered node sequences l before it i , where 1<=i<=r. For node c, this is {start, a, b}, and for node e, this is {start, a, b, c, d} and {start, b, a}. Then, L i ={l i}.

[0075] 7. Then calculate P i ←i All nodes in n are connected to 4 i , scan the graph and remove any shortcuts. For node c, P c ←={start, a, b}, and for node e, P e ←={start, a, b}.

[0076] 8. Repeat steps 4 and 5, but look forward instead of backward. For node c, P c →={d, e, end}, and for node e, P e → = {end} and {end}. Connect c to {d, e, end} and remove the shortcut, and connect e to {end} and remove the shortcut.

[0077] Figure 7 The output of the algorithm after processing the historical build sequence 500 of the part is shown in graph 600. Note that the first step in various examples always adds the {start} node 602 to the graph 602. In general, the resulting graph 600 may look different if the rows are ordered differently, so that newer information is prioritized. Furthermore, the algorithm in steps 4 and 5 can be modified. For example, steps 4 and 5 can be summarized as follows: If every n i When it appears, n j appears before it, then node n j In n i Before. This condition can be changed to treat the "optional" part differently. The condition is: "If whenever n j When it appears in a sequence, it is in n i Before, node n j In n i Before". Let I ← are all rows that appear in node n i All nodes before. Let I → are all rows that appear in node n i Then, for node n i , which is preceded by P ← =I ← \I → , and node n i In P → =I → \I ← Before.

[0078] To generate a graph 600 for automotive assembly (e.g., a precedence constraint graph), part number sequences are used while ignoring mounting parts such as nuts and bolts because these parts can be considered a "super part" along with the parts they support. In the example shown, five years of data are used (although other time periods can be used), and multiple sequences are provided for each year to generate the graph 600, given that multiple production line rebalancing occurs during a year. For new parts, in some examples, domain experts manually insert the parts in a valid sequence and generate a log. After several generations, an algorithm is used to perform optimization on the new parts. In some examples, with the precedence graph mined, when a user of the digital twin 102 drags and drops a task onto the Yamazumi board 218, the digital twin 102 shows the boundaries where the items can be placed. If constraints are generated for the new part, the optimization algorithm can rebalance the production line.

[0079] Thus, in various examples, the digital twin 102 is configured as a decision support tool in assembly line rebalancing. In some examples, once the prioritized constraint graph 400, 600 is generated, the user can choose to automatically rebalance the entire production line, which may be difficult without knowing all of the other constraints on the production line (e.g., light intensity, ergonomics, tool constraints, etc.), or to incrementally select the next optimization step to perform. In this case, based on the prioritized constraint graph 400, 600, all other factors that are not yet present in the digital twin 102 can be considered, where the next optimization step is not performed until the production line is balanced. In one example, using Figure 8 The user interface 700 and Figure 9 The user interface 800 shown is used to perform line balancing. In some examples, the user interfaces 700, 800 are displayed to the user side by side. However, different display configurations are envisioned. The user interface 700 displays the tasks on the production line in each unit 112, as well as the parts and tools that are present in each unit 112 and the possible mixes (e.g., different combinations of parts). The information is implemented as a diagram as described in more detail herein. When the user selects a task, the part, tool, and build option are selected. If a similar tool or part exists in another unit 112, the tool or part is also highlighted. Thus, the user can quickly determine the complexity of moving a task from one unit 112 to another. The user interface 800 shows the travel pattern 802 of each unit 112 based on the vehicle sequence provided. The user can choose to view, for example, Figure 4The cell layout view shown in FIG. 8 is used to optimize the cell layout, or the line view in the user interface 700 is used to optimize the cell layout. In some cases, if an operation is too long, the parts required for one cell 112 can be placed in another cell 112. Thus, the digital twin 102 allows for simultaneous balancing of the line and cell layout using the user interfaces 700, 800. Changes made to the line by moving the horizontal bar 702 (representing a work task) are reflected on the line, and the travel pattern line 802 is modified. The digital twin 102 moves all the tools and parts required for that task to the new cell 112.

[0080] In some examples, worker overload and disruptions are predicted. For example, predicted operator overload is approximated by performing a walk pattern analysis in each unit 112 individually. However, it should be understood that the analysis becomes more complex when two or more workers are attempting to install a part simultaneously. Moreover, in some examples, the walk pattern analysis assumes that travel is done in a straight line with no roadblocks in between. In some examples, the digital twin 102 facilitates predicting operator overload and disruptions by using path finding to simulate upcoming vehicle sequences. For example, some walking areas may be blocked, and the simulated worker 114 will find a path to the destination that does not include the blocked path.

[0081] Additionally, in some examples, worker contact detection may be performed. In some examples, if two workers 114 are close enough to each other, the worker that is about to make contact will send a message about the object that is about to make contact, thereby indicating an interference. Running the full simulation, the user of the digital twin 102 can obtain a report on the number of times the interference occurred, the duration of the occurrence, etc. In one example, during conditions that include contact events, all setup times are increased by a factor determined by the domain expert. Then, in some examples, the simulated walking pattern will deviate from the calculated walking pattern, but as the digital twin 102 runs the simulation, the simulated walking pattern is tracked and the interference area is highlighted. For example, as Figure 10 As shown in simulation 900 of , for example, if two workers 114 working on fuel truck 902 come into contact (e.g., come into contact with each other) while moving along path 904 during the simulation, contact between the workers is identified and an alert can be provided.

[0082] Figure 11 1 is a flow chart illustrating an example method 1000 for performing assembly line suitability analysis using a digital twin, such as may be performed by system 200, according to various embodiments. At operation 1002, data related to the operation of the assembly line is acquired. For example, as described in greater detail herein, one or more sensors 110 acquire data related to the movement of workers 114 within one or more cells 112 along the assembly line.

[0083] At operation 1004, a virtual environment of the assembly line is generated using the acquired data. For example, the virtual environment is generated as a digital twin 102 and defined using an assembly process model 214 synchronized with real-time assembly line data determined based on the acquired data (e.g., travel paths, travel times, etc.). The digital twin can use data acquired from one or more databases that describe the assembly process of the assembly line, vehicles to be assembled along the assembly line, the line layout of the assembly line, part locations, or a combination thereof.

[0084] At operation 1006, the tasks of one or more production line workers are simulated using real-time assembly line data, historical data, and sensor data, as described in greater detail herein. For example, assembly operations performed by one or more workers 114 within a cell 112 (e.g., collecting parts from a bin, obtaining tools for performing the operations, assembling parts to a vehicle, returning tools to a cart, etc.) are simulated. In some examples, the assembly line includes a plurality of cells 112, and the simulation includes simulating different cell layouts for one or more cells 112 in the plurality of cells 112, and performing a sensitivity analysis to identify a cell layout optimization that minimizes travel time.

[0085] At operation 1008, a determination is made as to whether a change has been received. For example, a simulated change is made to the unit 112, such as to change the travel path of the worker 114, to move one or more items in the unit 112, etc. As described herein, one or more user interfaces can be used to view and modify the simulation data and dynamically view the results (e.g., changes to tasks in a particular unit). For example, the simulation data includes approximating travel times for a set of tasks using data obtained from one or more sensors along the assembly line.

[0086] If it is determined that a change has been made, then in some examples, at operation 1010, the assembly line is rebalanced in real time based on the simulation to reduce the walking time required for one or more line workers 114 to perform tasks. For example, the line balance change and / or the corresponding walking pattern line is modified at operation 1010. In some examples, an optimization algorithm is used to rebalance the assembly line, wherein at least a portion of the assembly line is rebalanced using one or more precedence constraints for new parts to be assembled along the assembly line or new positions of one or more items within one or more cells. In some examples, a virtual environment of the assembly line is displayed with multiple cells, the multiple cells including multiple items represented as multiple boxes within the multiple cells and multiple walking patterns represented by multiple lines, and in response to receiving user input (at operation 1008), one or more of the boxes are virtually moved, or in some examples, one or more walking patterns are updated, including moving one or more of the multiple lines, to show the updated walking pattern, thereby causing the assembly line to be rebalanced at operation 1010.

[0087] If no changes have been received or after the assembly has been rebalanced, the method 1000 stops at operation 1012 .

[0088] Thus, one or more embodiments described herein use a digital twin 102 to provide assembly line optimization. For example, different parts (e.g., optional parts) can be accommodated in the model using a precedence constraint graph generated by an algorithm described herein based on set theory (the algorithm generates a precedence graph while prioritizing newer information). Because the digital twin 102 can call any optimization algorithm through the API 220, as users work online and save their work, the digital twin 102 is able to identify further optimization opportunities by calling these microservices. Thus, in some examples, the adaptability of the production line is continuously analyzed. Combining the interactivity and visualization of the digital twin 102 with the suitability analysis gives ALFA enhanced performance, for example, the performance allows for the integration of heterogeneous databases with IoT messages.

[0089] Unless otherwise expressly indicated herein, all numerical values ​​indicating mechanical / thermal properties, composition percentages, dimensions and / or tolerances or other characteristics when describing the scope of the present disclosure should be understood as modified by the word "about" or "approximately." Such modification is desirable for various reasons, including: industrial practice; material, manufacturing and assembly tolerances; and testing capabilities.

[0090] As used herein, the phrase at least one of A, B, and C should be construed to mean a logical (A or B or C), using a non-exclusive logical "or", and should not be construed to mean "at least one of A, at least one of B, and at least one of C."

[0091] In this application, the terms "controller" and / or "module" may refer to, be part of, or include: an application-specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinatorial logic circuit; a field-programmable gate array (FPGA); a processor circuit (shared, dedicated, or group) that executes code; a memory circuit (shared, dedicated, or group) that stores code executed by the processor circuit; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system on a chip.

[0092] The term memory is a subset of the term computer-readable medium. The term computer-readable medium, as used herein, does not encompass transient electrical or electromagnetic signals propagated through a medium (such as on a carrier wave); thus, the term computer-readable medium may be considered to be both tangible and non-transitory. Non-limiting examples of non-transitory tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tape or hard drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0093] The apparatus and methods described in this application may be implemented partially or completely by a special-purpose computer created by configuring a general-purpose computer to perform one or more specific functions embodied in a computer program. The functional blocks, flow chart components, and other elements described above serve as software specifications that can be translated into a computer program through routine work by a technician or programmer.

[0094] The description of the present disclosure is merely exemplary in nature and, thus, variations that do not depart from the essence of the disclosure are intended to be within the scope of the present disclosure. Such variations are not to be regarded as a departure from the spirit and scope of the present disclosure.

[0095] According to the present invention, one or more non-transitory computer-readable media storing processor-executable instructions are provided, which, when executed by at least one processor, cause the at least one processor to: acquire data related to the operation of an assembly line; use the acquired data to generate a virtual environment for the assembly line, wherein the virtual environment is defined using an assembly process model synchronized with real-time assembly line data; use real-time assembly line data, historical data, and sensor data to simulate tasks of one or more production line workers; and rebalance the assembly line in real time based on the simulation to reduce walking time for one or more production line workers to perform tasks.

Claims

1. A computerized method comprising: Acquire data related to the operation of the assembly line; generating a virtual environment of the assembly line using the acquired data, wherein the virtual environment is defined using an assembly process model synchronized with real-time assembly line data; simulating tasks of one or more production line workers using the real-time assembly line data, historical data, and sensor data; as well as The assembly line is rebalanced in real time based on the simulation to reduce travel time for the one or more line workers to perform the tasks.

2. The computerized method of claim 1 , wherein simulating the data comprises approximating the travel times for a set of tasks using data acquired from one or more sensors along the assembly line.

3. The computerized method of claim 2, wherein the one or more sensors comprises an imaging device comprising a camera, a laser sensor, or an infrared sensor.

4. The computerized method of claim 1 , wherein the virtual environment comprises a digital twin and the acquired data comprises data from one or more databases describing an assembly process for the assembly line, vehicles to be assembled along the assembly line, a line layout for the assembly line, part locations, or a combination thereof.

5. The computerized method of claim 4, wherein the one or more databases include a production line layout database, an assembly process database, a parts database, a factory IoT database, a vehicle CAD database, and a manufacturing CAD library database.

6. The computerized method of claim 1, wherein the assembly process model is synchronized with the real-time assembly line data using IoT messages.

7. The computerized method of claim 1 , wherein the assembly line comprises a plurality of cells, and the simulation comprises simulating different cell layouts for one or more of the plurality of cells, and performing a sensitivity analysis to identify a cell layout optimization that corresponds to minimized travel time.

8. The computerized method of claim 7 further comprising using an optimization algorithm to rebalance the assembly line, wherein at least a portion of the assembly line is rebalanced using one or more precedence constraints for new parts to be assembled along the assembly line or new locations of one or more items within the one or more cells.

9. A computerized method as described in claim 1, further comprising displaying the virtual environment of the assembly line, the virtual environment having a plurality of cells, the plurality of cells including a plurality of items represented as a plurality of boxes within the plurality of cells and a plurality of walking patterns represented by a plurality of lines, and virtually moving one or more of the boxes in response to receiving user input, updating the one or more walking patterns, including moving one or more of the plurality of lines to thereby show the updated walking pattern.

10. The computerized method of claim 9, further comprising using one or more APIs to cause the virtual environment to access one or more optimization functions to visualize optimal solutions for the one or more updated travel patterns corresponding to one or more different vehicle mixes along the assembly line.

11. A system comprising: a plurality of sensors configured to acquire movement data related to movement of workers at the assembly line, wherein the plurality of sensors include an imaging device including a camera, a laser sensor, or an infrared sensor; and an analysis system that receives the movement data and is configured to: Acquiring operational data related to the operation of the assembly line; generating a virtual environment of the assembly line using the acquired movement data and the operation data, wherein the virtual environment is defined using an assembly process model synchronized with real-time assembly line data, wherein the assembly process model is synchronized with the real-time assembly line data using IoT messages; simulating tasks of one or more production line workers using the real-time assembly line data, historical data, and the acquired movement data; rebalancing the assembly line in real time based on the simulation to reduce travel time for the one or more production line workers to perform the task; and The data is simulated by using the acquired movement data to approximate the walking time for a set of tasks.

12. The system of claim 11, wherein the virtual environment comprises a digital twin and the acquired data comprises data from one or more databases describing an assembly process of the assembly line, vehicles to be assembled along the assembly line, a line layout of the assembly line, part locations, or a combination thereof, and wherein the one or more databases comprise a line layout database, an assembly process database, a parts database, a factory IoT database, a vehicle CAD database, and a manufacturing CAD library database.

13. A system as claimed in claim 11, wherein the assembly line includes a plurality of cells and the analysis system is further configured to simulate different cell layouts of one or more of the plurality of cells, perform sensitivity analysis to identify the cell layout optimization corresponding to minimized walking time and display the virtual environment of the assembly line, the virtual environment having the plurality of cells, the plurality of cells including a plurality of items represented as a plurality of boxes within the plurality of cells and a plurality of walking patterns represented by a plurality of lines, and in response to receiving user input, virtually move one or more of the boxes, update the one or more walking patterns, including moving one or more of the plurality of lines, thereby showing the updated walking pattern.

14. The system of claim 13, wherein the analysis system is further configured to use one or more APIs to cause the virtual environment to access one or more optimization functions to visualize optimal solutions for the one or more updated walking patterns corresponding to one or more different vehicle mixes along the assembly line.

15. The system of claim 11 , wherein the assembly line comprises a plurality of cells, wherein the analysis system is further configured to rebalance the assembly line using an optimization algorithm, and wherein at least a portion of the assembly line is rebalanced using one or more precedence constraints for new parts to be assembled along the assembly line or new locations of one or more items within the plurality of cells.

Images