Interactive, intelligent production and maintenance execution agent

Abstract

Methods and systems are provided for scheduling and issuing interactive instructions for manufacturing of a product and product sub-assemblies. A project production engineering knowledge base including an interactive production portfolio is accessed, and a workplan specifying target times for completing one or more of the product and sub-assemblies is received. Responsively, a schedule for performing the assigned flow of steps (FOS) of each workstation is generated to meet FOS requirements of related tasks at other workstations. The assigned FOS is then provided at each workstation at the scheduled time, with feedback from each workstation indicative of completion of each step. The feedback is verified to before each subsequent instruction of the FOS is provided to the given workstation.

Description

INTERACTIVE, INTELLIGENT PRODUCTION AND MAINTENANCE EXECUTION AGENTFIELD OF THE INVENTION

[0001] The present invention relates to the field of interactive techniques for managing manufacturing and maintenance processes.BACKGROUND

[0002] In a modem manufacturing organization, data regarding the manufacture of parts and products must be available to people throughout the organization. Accountants, for example, need bills of materials (BOMs) to budget the manufacturing costs for products. Engineers need a more detailed engineering bill of material (EBOM) that includes not only the parts and subassemblies necessary to build products, but also more detailed information, such as tolerances, standards, and specifications. A further in-depth version of the EBOM, also referred to as a manufacturing bill of materials (MBOM) may be available for use in the actual manufacturing operations and may include descriptions of processing steps, required machinery, and packaging. A further type of BOM, also referred to as a service BOM (SBOM) may include details on maintenance requirements related to product sub-assemblies.

[0003] However, while all users of manufacturing data must have current documentation reflecting current products, each department often uses its own system for creating its own relevant documentation. Manufacturing changes required by one department must then be manually made to documentation used by each of the other departments. Moreover, a large set of procedure files must be produced for a production portfolio during the life-cycle of the project. However, due to the complexity of coordinating communication between the R&D, engineering, and production divisions, those documents are frequently incomplete, inaccurate, outdated or missing; leading to lower yields, recurrent rework, longer time to market and significant added costs.SUMMARY

[0004] Embodiments of the present invention provide a system and methods for generating step-by-step interactive instructions for manufacturing and / or maintenance of products and product sub-assemblies, as well as for associated processes, such as assembly,disassembly, testing, packaging, calibration, installation, fault isolation. Within a manufacturing setting, instructions provided at any given workstation are scheduled to suit timing requirements of related tasks at other workstations. The proper completion of each instruction is verified by preset methods, such as artificial intelligence-based visual inspection before a subsequent instruction is provided.

[0005] The system as described herein provides organizations with a set of capabilities not otherwise available. The set of capabilities includes:1. Consolidation of a production portfolio system used by all divisions— R&D, engineering, and production— enabling internal communication between all divisions through shared, validated, engineering model-based files.2. Consistency and standardization, with an up-to-date digital knowledge base of manufacturing information including all production portfolios.3. A generative engineering visual language model similar to generative artificial Al that is based on language models.4. Automatic generation of engineering procedural knowledge, based on Interactive Part Catalog Items (IPCIs), actions, steps, flow-of-steps (FOS), and procedural data with each object having an internal flow representation.5. A self-guided automatic and semi-automatic, proactive flow for the diverse users to train and interact with the system for engineering knowledge for applications including completion / validation / revision / reuse / updating / approval.6. Automatic generation of triggered execution code from engineering procedural knowledge to allow a full updatable, reusable, revised and validated, interactive production step-by-step guide.7. Automatic generation of product tree with automated check of the integrity and correctness of the parts and the procedures, shows statuses, and proactively prompts the user for completeness.8. Recommendations for production line structure, based on procedure / step contents and professions needed.9. An automatic document creator engine that collects all relevant data from the database and layouts all content in the right order, for a fully automatic pdf, word, or video document, while automatically creating a table of contents (TOC) with the matching page numbers. The document creator facilitates generation of a full document based on a wholeprocedure or based on selective content or updating a specific content by comparing the changes made on the database updating in the document only the changed content for optimized performance.BRIEF DESCRIPTION OF DRAWINGS

[0006] For a better understanding of various embodiments of the invention and to show how the same may be carried into effect, reference will now be made, by way of example, to the accompanying drawings. Structural details of the invention are shown to provide a fundamental understanding of the invention, the description, taken with the drawings, making apparent to those skilled in the art how the several forms of the invention may be embodied in practice. In the accompanying drawings:

[0007] Fig. 1 is a high-level block diagram showing production portfolio generation modules (WizGen and WizPlan) and production execution modules (WizShield and WizField) of a system for manufacturing and field support, according to embodiments of the present invention; and

[0008] Fig. 2 is a schematic, block diagram of a system for intelligent scheduling and production execution in a manufacturing facility, according to embodiments of the present invention.DETAILED DESCRIPTION

[0009] Examples of the present invention provide a system that implements real-time, step-by-step, interactive production guidance based on processes defined in production portfolios. Execution guidance may include safety, assembly, dis-assembly, rework, test, training, fault isolations, maintenance, packing, unpacking, tutorial, install, calibration, removal. Production is controlled by step-by-step execution and with result verification at each step, which may be implemented at diverse facility types and at distributed locations.

[0010] Fig. 1 is a high-level block diagram showing production portfolio generation modules and production execution modules of a system 20 for manufacturing and field support, according to embodiments of the present invention.

[0011] The system includes four main modules. An intelligent product portfolio generator 30 (also referred to herein as “WizGen”) produces product portfolios includingproduct bills of material (BOMs), engineering instructions for production, maintenance and repair, stored in a database accessible by the other modules. Data is entered to the system by production people with various project roles, including the project manager 32 and documentors 34.

[0012] A production line planner module 40 (also referred to herein as the production planner, or “planner,” or “WizPlan) oversees allocating specifications of the product portfolio generated by the portfolio generator 30 and stored in the system database, to virtual production line‘s workstations, while splitting engineering procedures to relevant workstation’s procedures based on each workstation constraints - tools, materials, and jigs. Each virtual production line can represent a whole project or a specific IPCI part’s assembly procedure. Each virtual production line’s size depends either on a real production line and then the number of workstations is predefined, or by creating a new production line based on the needs and constraints that are expressed in the procedure structure.

[0013] The allocation of procedures or procedure splits to virtual production line workstations can be done either automatically by the production planner 40, or manually by the planner. The production planner 40 analyzes each procedure structure and step types to find any information regarding complexity of all steps - long time periods steps (e.g., glue steps that need waiting) and specially needed instruments (e.g., oven, testing equipment...) and other criteria, such as production quantities, expected timelines, number of employees, number of shifts, each one of them can be input or predicted. This analysis helps the production planner 40 to understand the procedures and recommend where the manual allocation should be done.

[0014] An interactive scheduler and production execution module 50 (also referred to herein as WizShield) determines schedules for presenting production instructions at workstations 52 of workstations determined by the production planner, and then presents those instructions, step-by-step, at the workstations 52, for both manual and automatic (e.g., robotic) product manufacturing. The production execution module verifies the correct execution result for each step, while generating automatic checklists reports at a step level, for review by a line manager 54. Specifications for the scheduler and production execution module are created by the production planner 40 and stored in the system database.

[0015] When a virtual line planning is ready for testing in WizShield the production planner or the project manager can generate a snapshot of the project / procedure planning.Each ready virtual line snapshot in WizShield can be used as a base for preparing a workorder scheduling workstation's tasks by the WizShield line manager. Each snapshot contains a list of all allocated procedures for each workstation and the list of the needed instruments. - tools, materials, and jigs.

[0016] An interactive field repair and maintenance execution module 60 (WizField) presents engineering instructions, step-by-step, to support maintenance tasks and repair processes, field repair and maintenance execution module guides a technician 62 through steps of assessing a fault, performing corrective actions, verifying the correct execution result for each step, and generating automatic checklist reports detailed to the level of individual steps. Specifications for the field execution module are created by the portfolio generator 30 and are stored in the system database.

[0017] Fig. 2 is a schematic block diagram of module 50 for intelligent scheduling and production execution in a manufacturing facility, according to embodiments of the present invention. Sub-modules of the scheduling and execution module 50 are divided between modules (i.e., sub-modules) of a scheduler system 210 and modules of an interactive production execution system, referred to as the Runner system 240.

[0018] The scheduler 210 and Runner 240 get input from the production database 270, which includes the output from the production portfolio generator 30 and from the production planner 40, as well as statistics (such as production statistics and analytics for all lines that run in a facility). The scheduler and Runner also gets input from external content 272 typically by application programming interfaces (APIs). Such content may include enterprise resource planning (ERP) systems, customer resource management (CRM) systems, providing customer and project information, product lifecycle management (PLM) systems, and manufacturing execution systems (MES), to which, for example, the Runner may provide production test results. Work orders, for example, may be provided by PLM or ERP systems, or added manually. External content may include the following types of data, as well as additional types not listed:• BOM, EBOM, MBOM• CAD in various representation formats model and model images• DOC, DOCX, PDF documents• JPG, JPEG, PNG, TIFF, GIF, SVG images• MP3 videos• DWG, DXF drawings• Procedures and sub-assemblies• Log files, database formats, spreadsheets• Manual ECO, MRB

[0019] Manually added specifications, such as work orders, may be provided by user interfaces 274, such as an administration dashboard of the module 50.

[0020] A communications interface 276 may receive signals and textual details from instruments in the facility, as well as connect with diverse input devices such as still image and video cameras, phones, microphones, barcode readers, 2- and 3D-scanners, sensors, touchpads and other devices, to receive feedback from the work floor, workshop, lab and / or from maintenance and repair operations in the field.

[0021] Scheduler

[0022] The scheduler 210 typically operates by both automatic and manual processes.

[0023] Automatic scheduling aligns tasks to workstations according to the allocation performed in the planner, and according to work order requirements. The work order scheduling module 220 receives work orders defining how many product assemblies should be manufactured or otherwise provided by a specific date, allowing a line manager to schedule related tasks. A work order may have additional fields, such as details of a client to receive the work. Additional details may include the organizational unit responsible for the product and the purchase unit and responsible employee. A maintenance review board (MRB) system may provide specifications of changing parts / sub-assemblies and whether parts can be fixed and reused or must be deprecated.

[0024] In addition to assigning work to workstations, at which work may be performed automatically (e.g., robotically) and / or manually, the scheduler may specify specific employees for a task according to their qualifications and current certifications. At the time of task allocation worker certifications are checked. The scheduler is programmed in advance and works in a separate editing mode that does not interfere with the production line that is running.

[0025] Additional requirements for the scheduler include tracking of resource availability for a given scheduled task. The scheduler will inform selected roles of the need for parts, components, tools, materials, and jigs that should be available for inspection within a given period prior to production (the time period being defined, for example, by aproduction line manager). If resources are not available when expected, the scheduler will perform rescheduling of a target production task. Tasks are decomposed to lower assemblies; when all resources are available, the tasks’ requirements are marked as active, meaning ready for assembly. Tasks that do not meet all requirements will display an unready status and send timed notifications about expected readiness date to the line manager. The scheduler may allocate jobs to prioritize preparation of resources including materials, tools, jig preparation, installation, calibration, and testing before their use is needed in a required assembly task. The line manager can manage the status of each workstation he oversees.

[0026] The output of the scheduler is timed tasks allocated to workers and workstations, tasks that can be reviewed according to their allocated time, date, or period, such as a week or a month. These events send alerts to diverse users according to the policy defined in the scheduler. Scheduled task events become active when their scheduled time arrives.

[0027] Additionally, scheduled work orders may specify periodic maintenance routines. The field module described above provides direct access to event-driven maintenance and fault isolation routines, as explained below.

[0028] An additional module of the scheduler is the Uploader and Sync 222. As described above, the planner creates production lines for workstations including step-by-step instructions, based on instructions generated in the portfolio generator. Planner output includes assignment of tools, consumables, and jigs to workstations, creating a full production line.

[0029] When the production planner generates a new snapshot of a project's procedures, then it is available for the LM to use it for scheduling for a work order (for a dry-run). No need for any other action, it is ready to use. When the snapshot is in a separate installation, then the planner generates a new virtual production line and exports the snapshot from one installation to another. The line manager can then upload the new production line version and assign logical workstations to physical workstations in the floorplan by connecting to their IP address. The IP-address matching process allocates physical workstations to the specific instructions defined for virtual workstations by the planner. (The production line manager can select the IP address from a pull-down menu for each virtual workstation in the production line when IP addresses are known. The production line manager gets a physical layout of the IP addresses in which he overlays the virtual line workstations. The matching between the virtual line’s workstation and their corresponding IP address is not changedwhile importing new lines unless a workstation is out of order or there are physical changes or upgrades in the production facility that affects the physical workstation. When a workstation is out of order, the line manager can use a temporary workstation which is currently available and has the same tools, jigs, and consumables as the faulty workstation, by setting a temporary IP address of the faulty workstation).

[0030] Each time the production planner finishes and generates a new planned production line, a snapshot of the new line is created and uploaded to the database, triggering a new execution of the scheduler. Changes in the line layout are triggered by any of the following:1. Changes in one or more procedures contents, without any changes in the planner line layout.2. Changes in the line layout or with station procedure allocations, without any changes in the project procedure content.3. Changes in both project procedure content and line layout and station procedure allocations.

[0031] Uploading the new production line version can be done in the following ways:1. When scheduler database schema is located on the same database as the project portfolio, the new virtual line snapshot is generated by the planner completing the relevant DB schema. The line and production managers get notification of the new virtual line snapshot waiting to be allocated to the physical workstations.2. When the scheduler database schema is in a separated database, the uploading of a new virtual line snapshot is done by uploading a file of the new snapshot into the database schema. The file includes a snapshot id and the production line version number, with executable step-by-step procedures together with the images to be shown and additional metadata.

[0032] The newly uploaded snapshot is created under an Illustrated Part Catalog Item (IPCI), which is the main object of the project portfolio. The IPCI includes workflows that are to be executed step-by-step, with each step typically including a verification step, with verification feedback from a workstation confirmed by the Runner system, ensuring that the step is completed correctly before a subsequent step can be executed. The workflows are typically stored as linked lists of stepwise procedures.

[0033] Each time the planner installs an IPCI project snapshot, it notifies the scheduler of any workstation changes. The production line manager will also get a notification and a changes report made from previous snapshots instructions or configuration in workstations. If any workstation configuration is included, a notification will be sent to the line manager requiring the line manager verification that a workstation is ready before the production line scheduler can assign a work order and its tasks. The line manager will also get a notification and a change report made from previous snapshot instructions or in workstations.

[0034] The production execution module enables draft (OPEN) snapshots as well as approved ones. The draft snapshots, commonly known as DRY RUN, include instructions that should be checked in the assembly line for correctness, changes, step order, jig fitness and any other possible fix in the instructions content. A line manager and line worker can add comments for changes that will be managed through the portfolio generator. Following the updated instructions, changes are propagated to the chain of approvers. Upon the final approver a new (CLOSED) version is created, and notifications are sent to authorized users. Subsequently, a new snapshot is generated by the planner, generating an executable FOS step by step format.

[0035] Input snapshot structure

[0036] A project’s virtual line snapshot that is generated from the planner to the production execution module contains a full line of workstations with allocated procedures to each workstation. All procedure’s steps that were saved in the portfolio as raw step content are converted into ready-to-use images in a dedicated project images folder, and every step references the relevant image file for instant loading in the Runner module.

[0037] Data in the database of the scheduler and Runner tables, generated by the planner, may include the following:1. Content tables of the portfolio contain all project IPCI parts with all their procedures, flow of steps (FOSs), as well as steps converted to ready-to-use images.2. Additional planner output of virtual line and station tables with the allocated procedures for each station.3. Line tables: a virtual representation of a physical line to create a product from a project snapshot. It contains all stations and their layout on the production floor.4. Station table: a virtual representation of a physical workstation in the production line. It contains a list of all instruments: tools, jigs and consumables needed at the workstation and a list of the procedures that can be executed on the workstation.5. Workstation reference table: a list of references for the instruments needed at each workstation.6. Station FOS table: a list of references of the procedure’s flow of steps (FOSs) to be executed on each workstation7. Event table: definitions of scheduled events to run on each workstation. Events can be, for example, when to mn safety instructions or generic guidelines.

[0038] Additional modules of the scheduler may include:• Self-guided workflow module 224• Statistics and analytics module 226• Smart step counter 228• API manager 230• Help and tutorial 232• MRB Manager 234

[0039] Runner

[0040] The Runner 240 dispatches each scheduled procedure at the scheduled time, that is, it provides each scheduled procedure to a given workstation 280 (either a manufacturing location for completing a production or maintenance task). In case a workstation became available earlier than planned, the line manager may release the execution of a scheduled task before its scheduled time, which requires that the line manager update the planner timeline. The Runner gets from the planner all the necessary information and content required for executing production procedures. The Runner may be an interactive smart agent that can run standard FOSs and step types defined in the product portfolio. Several components execute specific step types in standard sequences that include diverse types of instructions such as assembly, disassembly, rework, fault isolation, maintenance, tests, calibration, installation, and more. The content is typically an object defining one or more steps, wrapped in an object structure defining their flow (Flow of Steps -FOSs).

[0041] The Runner typically starts a procedure by retrieving an identification of the IPCI to be run. Typically, an assembly worker accesses a workstation to initiate execution of the Runner, which then provides the procedures for each step of a scheduled task. The Runnermay run as an interactive smart agent at the workstation, providing atomic steps to be executed in a specific flow. For each step, all the necessary knowledge for correct execution may be displayed.

[0042] The Runner may provide procedures that are manually performed by a worker to achieve a digital “twinship” between a designed IPCI and its physical embodiment.

[0043] The first FOS to be executed when a worker starts the task execution is typically a sequence showing a facility’s guidelines and safety procedures. This helps to reduce accidents and disasters during the production process and during maintenance stages.

[0044] There are several levels of guidelines and safety instructions:• Procedure / step internal ones: written by the documenter on a specific step / s, such as “Ensure not to bend the fiber cable while threading it!”• IPCI guidelines- written by the IPCI documenter (can come with the IPCI from its manufacturer) on that specific part / tool / jig / consumable such as a piston, knife, glue, or drill.• Project guidelines: written by the documenter as dictated by the project manager or the senior engineer, such as “Using a hammer in any step of this project is forbidden!”• Manufacturer guidelines: written by the documenter as dictated by the manufacturer to all the projects as consistent guidelines policy, such as: “Return JIGs to their place after use.”• Production floor guidelines: written by the production floor manager to make sure all workers work with the local guidelines and safety instructions, such as: “No entrance without wearing helmets and goggles!”

[0045] The different types of guidelines and safety instructions are presented as follows by the Runner module.

[0046] Internal guidelines are defined in the content of procedures generated by the planner and are shown to the worker every time he gets to that procedure, in every iteration of execution of the procedure.

[0047] The IPCIs, project, and manufacturer guidelines are also received from the planner but are shown to the worker as defined in the event frequency referenced in the line and / or station DB records. The line has the default value declared, and in every station, that frequency can be overwritten because of specific needs. The default frequency is every newprocedure session. If a worker starts a different procedure, it means a new session. If he takes a break, it means starting a new session.

[0048] Typically, guidelines are created by the documenter in the portfolio generator. These guidelines and safety instructions are added to each project with the generation of the line procedures in the planner.

[0049] As described above, the Runner provides procedures to each workstation according to the assigned FOS, to achieve a digital twinship between the designed IPCI and its physical embodiment, the twinship being implemented by the twinship module 250 of the Runner. The Runner executes each scheduled procedure at the time it was scheduled. If a workstation is available earlier, a line manager may update the scheduler timeline.

[0050] The Runner gets from the planner the necessary information and content required for executing each production procedure. Several components execute specific step types in standard sequences for each of the different types of procedures, such as assembly, disassembly, rework, fault isolation, etc. The Runner typically first gets identification of an IPCI to be produced by a given workstation.

[0051] As described above, typically each procedure of each FOS presented at a workstation includes a verification action by a validation and verification module 252, whereby either a worker confirms manually that a result is correct, for example that an image result is accurate, or an automated verification is performed, for example by automated visual inspection.

[0052] Additional modules of the Runner may include:• Interactive catalog 254• Interactive inspection 256• Interactive step-by-step procedures 258• Interactive checklist ATP steps 260• Interactive drawings 262• Training 264

[0053] Database

[0054] The database 270 links the modules of system 20. The database includes subdatabases of various types including relational databases providing SQL access and NoSQL databases for document management. Additional data included in database 101 include:• Frozen snapshots of the database and snapshots database tables.• All data for current open projects per each installation (the “main editing database”).• Repositories such as 3D models, 2D views and pictures, algorithm generated views, machine-vision and machine learning (ML) models, videos, animations, encoded files, text files, and log files.

[0055] The main editing database and the version databases tables and table relations facilitate the generator capabilities of the system, as well as document creation, for example creation of documents in DOCX and / or PDF formats. Database tables are defined in groups including:• Setup tables: contains configuration, guidelines, and instruments.• Generic tables: raw images and projects• Communication and statistics tables: notes, content status, and statistics• Version control , releases and snapshot tables• Catalog tables: IPCIs and relations between them• Procedures tables: all procedure content and content relations• Drawings tables• User control and permission tables

[0056] Input to the database, as well as the output from the database, includes all project content, which is stored as records in the database tables.

[0057] The production execution and field maintenance share a separate set of database tables with the same database schema. The tables are divided into several categories, as follows:

[0058] Content tables that were generated by the portfolio generation module are used in the same format in the production execution and field modules, with the ready-to-use images acquiring an additional layer for interactive manipulations on the workstation interface.

[0059] Content tables that are generated in the planner module are also used as-is by the production execution module, including:1. Line table: The line table is identified by its id field. Its other fields are:• name: a readable identification: usually the name of the IPCI to be assembled and it can be modified by the line manager.• status: non-revised / revised - Non revised indicates that the instructions steps and their execution order is still in a draft mode, and should be tested on the production floor, updated if needed and sent to the list of approvers. Only after final approval is line content (final instructions to be executed on the production line workstations) closed and its status changed to “revised.”• projectld: a reference to the project content created in the portfolio generator (WizGen).• layout: a JSON structure with the workstations’ virtual locations on the line and the relations between them• safetyEventld: a pointer to a record in the Event table. This field has the default safety instructions frequency for all workstations in the line. The line has the default value declared, and in every station that frequency can be overridden by the production line manager because of specific needs. The default frequency is every new procedure session. If a worker starts a different procedure, it means a new session. If he took any type of break, it means starting a new session.• genericEventld: a pointer to a record in the Event table. This field has the default generic guidelines frequency for all workstations in the line.2. Station table: A table for each workstation, identified by a lineld and id fields. Other fields are:• name: a readable identification that might be updated by line manager.• status: the status field may have different states, with a default value when first instantiated being “inactive.” It will become active at the time a workstation starts to perform an assigned task. Other status values may include halt (pause) when a station has been stopped; break, when a worker left for a lunch or other planned activity; and fault, when a station has a problem while running the step-by-step instructions; and release.• safetyEventld: a pointer to a record in the Event table. This field enables overriding the default safety instructions frequency for this workstation.• genericEventld: a pointer to a record in the Event table. This field enables overriding the default generic guidelines frequency for this workstation.3. Station References table: The Station References table enables adding an IPCI reference to a specific workstation identified by the lineld and its stationld fields. The IPCI reference is identified by the IPCI’s catalog number.4. Station FOS table: The Station FOS table is used for saving the relation between the line workstation and all the FOSs allocated to it. Each record may also include the next station FOS record that will get the output of the current record, which enables splitting the output of a FOS on the station to multiple stations. The record is identified by the id field. Its other fields are:• lineld: index that links to the line.• stationld: index that links to the station.• fosld: the id of the allocated FOS.• beginFlow: id of the first step in the FOS that must be executed. This enables starting from the middle of the FOS.• endFlow: id of the last step in the FOS that must be executed. This enables ending in the middle of the FOS.• stationFosId: the link to the output stationFos record, to enable splitting the output.5. Event table: The Event table is used to declare frequency of showing a type of content. It is identified by the eventld field. Its other fields are:• type: safety instructions / generic guidelines type flag.• frequency: session / daily / weekly / monthly: session is the default value.

[0060] Content tables that are added in the WizShield production execution module, including:1. Factory table: The factory table enables managing multiple production floors in the factory. It is identified by its id field. Its other fields are:• name: a readable identification that can be modified by the factory manager.• status: Status might have different states. Its default value when its first instantiated is inactive. Other states are: a) Active: when at least one floor has become active. b) Stop: when at least one floor has stopped.• location: has information about where the factory is located.2. Floor table: The floor table enables managing multiple physical production lines on different floors. It is identified by its id field. Its other fields are:name: a readable identification that can be modified by the factory manager. status: Status might have different states. Its default value when its first instantiated is inactive. Other states are: c) Active: when at least one physical line has become active. d) Halt (pause): when a line has been momentarily stopped by the worker. e) Break: when a worker leaves for lunch or other planned activity. f) Fault: when a station has encountered a problem while running the step- by-step instruction. It might be due to equipment malfunction or worker assembly mistake. g) Stop: when the line manager decides to immediately stop a station. h) Release: Stop and Fault statuses can be changed only by the line manager.• location: has information about where the line is installed. It enables management of different configurations of the same production lines according to the physical constraints of specific location facilities.3. Physical line table: The physical line table represent the real line on the production floor that will be connected to a virtual line by the line manager who schedules the task for each work-order.4. Physical workstation table: The physical workstation table represent the real workstation on the physical line that will be connected to a virtual workstation by the line manager who schedules the task for each work-order. The tasks are allocated to the physical workstation.Each workstation has an IP address in the production floor that represents a physical asset in the facility. The virtual workstation index that has been assigned by the Planner should be matched with the IP address for verifying correct allocation of work procedures to the workstations in the production floor line.

[0061] Additional tables used by the production execution module may include a work order table, task table, task user relation table, a WizShield execution checklists table, an MRB table, and a products table.

[0062] Additional field module tables may include: a ticket table, an issue table, an issue missing IPCI table, a maintenance execution table, a Field execution checklists table, a customer company table, and a product instance table.

[0063] Procedures

[0064] A procedure is a conceptual process that describes any kind of engineering process performed on an IPCI. The IPCI is identified by a single part number which can be comprised of other IPCI’ s. As explained before procedures are created in the WizGen module and generate executable digital code in WizPlan. "New" processes come out of WizPlan, (which may be different from the ones in WizGen), where sections of the WizGen procedure are split to different workstations and in every station the WizGen procedure section becomes a separate procedure for the relevant station.

[0065] Each procedure is implemented by one FOS, which can be linked to other FOSs, while using conditions and loops between the steps and FOS links. The trigger of the execution path is based on the input value entered.

[0066] There are two classes of procedures that enable performance of specific tasks as follows:A. Internal Procedures: Series of FOSs that apply to a specific IPCI and all its descendant IPCIs. They include assembly, disassembly, rework, test, fault isolation symptoms and maintenance.B. External procedures: Series of FOSs that apply on any IPCI. They relate to theIPCI as a black box and provide series of FOSs that are performed on it. They include safety instructions, calibration setup, install, removal, replace, packing, unpacking, manual.

[0067] Training is a type of procedure that is both external and internal. That is, its meaning applies to each one of the procedures, either internal or external.

[0068] Before starting any type of procedure a mandatory series of FOSs comprised of informative and action steps must be performed for the purpose of implementing safety guidelines. The IPCIs, project, manufacturer and production floor guidelines are shown to the worker when starting a new session (meaning different procedure or after returning from a break), or as defined in the event frequency referenced in the line and / or station DB records.

[0069] Procedures may be of any of the following types:

[0070] Assembly: It is the core procedure of an IPCI. It is a procedure with a defined flow of steps that guides how to assemble a part. It will include parts and instruments inspection, informative and action steps with input values such as numeric values for input validation, links for flow implementation and results for assembly correctness assessment.After recursively repeating the assembly process with different IPCI each time, the company accomplishes the assembly of the whole product. The process to perform an assembly procedure is defined as follows: i. Parts Inspection FOS - Scanning the Barcode and serial number (if appears) of each item that is indexed in a single or series of BOM images (including its descendants if appear). Alternatively, runner can manually insert IPCI code and when prompted - also serial number.When more than one item per IPCI is required (as in the case of bolts or screws) runner will be prompted to insert the item quantity per BOM image.Each assembly IPCI procedure can have only 1 Parts Inspection FOS that show all relevant IPCIs for that procedure. Each Parts Inspection FOS will have a list of action test steps followed by 1 or more informative image steps that show the IPCI’ s explosion. ii. Instruments Inspection FOS: Scanning the Barcode or manually inserting the IPCI code of each tool, material and jig required for performing a procedure.Each assembly IPCI procedure has only 1 Tools Inspection FOS that shows all relevant tools for that procedure. Each Tools Inspection FOS has a list of action test steps.Each assembly IPCI procedure has only 1 Materials Inspection FOS that shows all relevant materials for that procedure. Each Materials Inspection FOS has a list of action test steps.Each assembly IPCI procedure has only 1 Jigs Inspection FOS that shows all relevant jigs for that procedure. Each Jigs Inspection FOS has a list of action test steps. iii. Perform informative, action, check, split, string, barcode steps to build assembly FOS for the IPCIs that comprises the assembly. Assembly FOS can call other FOS to perform an identical operation, containing a list of part catalog numbers to be replaced in a twin assembly or an opposite assembly operation, containing a list of IPCI numbers and image manipulations instructions (e.g., mirror, scale...) needed to create the opposite assembly set of steps for example two opposite airplane wings.iv. Every step or short series of steps prompts for a check result or an input value to be assessed. Success will lead to the next step. Failure will prompt rework.

[0071] Disassembly: A procedure that guides the runner through a reversed list of steps of the assembly list of steps, while replacing each action with its opposite action including parts and instruments inspection, informative, action with input values and links. In the disassembly reversed step list, for each step which has a result, the new step should be vice versa. That is, the result image should be the step image and the step image should be the result image. For actions that have no opposite (like “Perform”) or when the opposite action is not 100% clear, the user might get recommended, closely-related actions and will typically fix the action manually. The disassembly might be a FOS in a rework procedure.

[0072] Rework: a defined flow of link steps that link to specific steps or FOSs in assembly / dis-assembly procedures for fixing or replacement informative and action steps with control buttons. Rework will be triggered by a change in an IPCI with all its relevant procedures or by a failure in a FOS test / result.

[0073] Maintenance: a defined flow of FOSs and steps that may also have links to specific FOSs in assembly / dis-assembly procedures. This procedure allows to choose parts and instruments inspection, informative and link, while actions are typically designed with control buttons. Maintenance aims at guiding the user through the expected procedure via text, image or video. This procedure is used to define a frequency (schedule) how often it should be run in production. Maintenance can be applied according to three possible events:1. A time event - Mandatory scheduling - a frequency (schedule) defined by the engineer regarding to when the maintenance procedure should be applied. It fits periodic maintenance.2. A utility function event. Optional. The utility function can be defined by the possible damage estimated by the cost of the possibly damaged parts + waiting time for replacing parts + cost of replacing parts + cost of work + the idle time for repairs + an estimated factor of additional damages (safety or others) divided by the cost of replacement parts + cost of work + plus idle time for repairs. A threshold is calculated and in a case the function value is beyond the defined threshold the maintenance procedure is recommended. Scheduling is according to engineer selection. In the case of data collection, function can be determined by an ME algorithm.3. A broken IPCI event: in that case procedure is the same as rework.

[0074] Test - A defined flow of informative and action steps or FOS to check. Input values can be yes / no, or other values as described in the action step type. It can be formatted as a checklist of individual tests as in an acceptance test. Test can also be an ATP (automatic test plan) while interconnecting (getting input from sensors or test equipment or instruments in the test station). More complex tests may also have link steps, parts, and instruments inspection.

[0075] Fault Isolation: A fault isolation engine displays a step-by-step test map based on a set of iterative informative and action steps with user input to identify the fault followed by the corrective fixing path until the symptom disappears. A fault isolation symptom of a specific assembly is identified by the highest frequency defined by the engineer or automatically assessed by machine vision, Al and / or ML. The fault isolation symptom identifies a possible fault and the actions / instructions to be performed. Thereafter the fault isolation symptom is assessed. If it does not disappear, the next highest frequency fault isolation symptom will be checked until fault isolation symptom is removed.

[0076] Each fault isolation symptom can be described by specific text, image, video, sensor, or test value. Each fault isolation symptom is declared under an IPCI and relates to it. Each IPCI may have many fault isolation symptoms. The action steps lead a user through a diagnostic process to one of the (action) conclusion steps to be checked manually by the user or automatically by machine vision or another automated algorithm. Each conclusion step is used to indicate to the user what he needs to do to fix the cause of the symptom. The conclusion step can consist of:• A message to the user.• The symptom reason that caused the symptom to happen.• The fix description with links to the relevant procedures for fixing.• The indication can be performed via text image or video if requested so by the user.

[0077] User might be asked to add input in varied formats. Input may include original images / videos, text, or test value and all test / conclusion steps and other content added manually by a user as uploaded files. Additionally input from sensors might be automatically collected.

[0078] Calibration setup of a part / tool / jig is also a kind of tutorial that is used for resetting it to the manufacturer definitions.

[0079] A generative engineering visual language structures the generation of engineering procedural knowledge. The following table is a glossary of terms used in the system, as described above.

[0080] EXAMPLES

[0081] An example of the present invention is a method of scheduling and issuing interactive instructions for manufacturing of a product and product sub-assemblies, including the following steps. A first step includes accessing a project production engineering knowledge base including an interactive production portfolio that includes a definition of a flow of steps (FOS), where the FOS defines sequences of instructions for manufacturing the product and sub-assemblies, and where the project production engineering knowledge base also includes one or more required resource definitions of tools and materials that are required at each of multiple workstations at which the FOS are performed. The project production engineering knowledge base also may include specifications of workstation assignments, assigning the FOS and definitions to each of multiple workstations. A second step includes receiving a workplan specifying target times for completing one or more of the product and sub-assemblies.

[0082] A third step includes generating the workstation assignments, according to the product portfolio, and generating a workplan for performing the assigned FOS at each workstation. The FOS schedule for each workstation is generated to meet FOS requirements of related tasks at other workstations. Next, a first step of the assigned FOS is provided to each workstation at the scheduled time. During each step of the FOS, implemented by the workstation, a list is received from the workstation indicating the actions performed, tools, materials, and jigs used, and feedback indicative of completion of the step, and correctness of the list is verified before a subsequent instruction of the FOS is provided to the workstation.

[0083] It will be readily apparent that the various methods and algorithms described herein may be implemented by appropriately programmed general purpose computers andcomputing devices. Typically, a processor (e.g., one or more microprocessors) will receive procedures from a memory or like device and execute those procedures, thereby performing one or more processes defined by those procedures. Further, programs that implement such methods and algorithms may be stored and transmitted using a variety of media in several manners. In some embodiments, hard-wired circuitry or custom hardware may be used in place of, or in combination with, software procedures for implementation of the processes of various embodiments. Thus, embodiments are not limited to any specific combination of hardware and software.

[0084] A "processor" means any one or more microprocessors, central processing units (CPUs), computing devices, microcontrollers, digital signal processors, or like devices.

[0085] The term "computer-readable medium" refers to any medium that participates in providing data (e.g., procedures) that may be read by a computer, a processor or a like device. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical or magnetic disks and other persistent memory. Volatile media include dynamic random-access memory (DRAM), which typically constitutes the main memory. Transmission media include coaxial cables, copper wire and fiber optics, including the wires that comprise a system bus coupled to the processor. Transmission media may include or convey acoustic waves, light waves, and electromagnetic emissions, such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer- readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EEPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.

[0086] Various forms of computer readable media may be involved in carrying sequences of procedures to a processor. For example, sequences of procedure (i) may be delivered from RAM to a processor, (ii) may be carried over a wireless transmission medium, and / or (iii) may be formatted according to numerous formats, standards, or protocols, such as Bluetooth, TDMA, CDMA, 3G.

[0087] Where databases are described, it will be understood by one of ordinary skill in the art that (i) alternative database structures to those described may be readily employed,and (ii) other memory structures besides databases may be readily employed. Any illustrations or descriptions of any sample databases presented herein are illustrative arrangements for stored representations of information. Any number of other arrangements may be employed besides those suggested by tables illustrated in drawings or elsewhere. Similarly, any illustrated entries of the databases represent exemplary information only; one of ordinary skills in the art will understand that the number and content of the entries can be different from those described herein. Further, despite any depiction of the databases as tables, other formats (including relational databases, object-based models and / or distributed databases) could be used to store and manipulate the data types described herein. Likewise, object methods or behaviors of a database can be used to implement various processes, such as the described herein. In addition, the databases may, in a known manner, be stored locally or remotely from a device that accesses data in such a database.

[0088] The present invention can be configured to work in a network environment including a computer that is in communication, via a communications network, with one or more devices. The computer may communicate with the devices directly or indirectly, via a wired or wireless medium such as the Internet, LAN, WAN or Ethernet, Token Ring, or via any appropriate communications means or combination of communications means. Each of the devices may comprise computers, such as those based on Intel or similar processors, which are adapted to communicate with the computer. Any number and type of machines may be in communication with the computer.

[0089] It is to be understood with respect to the above description that although various features of the invention may be described in the context of a single embodiment, the features may also be provided separately or in any suitable combination. Conversely, although the invention may be described herein in the context of separate embodiments for clarity, the invention may also be implemented in a single embodiment. Certain embodiments of the invention may include features from different embodiments disclosed above, and certain embodiments may incorporate elements from other embodiments disclosed above. Meanings of technical and scientific terms used herein are to be commonly understood as by one of ordinary skill in the art to which the invention belongs, unless otherwise defined. It is to be understood that the invention can be carried out or practiced in various ways and that the invention can be implemented in certain embodiments other than the ones outlined in the description above. Method steps associated with the system and process can be rearrangedand / or one or more such steps can be omitted to achieve the same, or similar, results to those described herein.

Claims

CLAIMS1. A method of scheduling and issuing interactive instructions for manufacturing of a product and product sub-assemblies, comprising: accessing a project production engineering knowledge base including an interactive production portfolio including a flow of steps (FOS) defining sequences of instructions for manufacturing the product and sub-assemblies, and including one or more required resource definitions of tools and materials required at each of multiple workstations at which the FOS are performed, and further including specification of workstation assignments, assigning the FOS and definitions to each of multiple workstations; receiving a workplan specifying target times for completing one or more of the product and sub-assemblies; generating, according to the product portfolio, the workstation assignments, and the workplan, a schedule for performing the assigned FOS of each workstation, wherein the FOS schedule for each workstation is generated to meet FOS requirements of related tasks at other workstations; and providing to each workstation a first step of the assigned FOS at the scheduled time; receiving from the workstation, during each step of the FOS implemented by the workstation, a list indicating the actions performed, tools, materials, and jigs used, and feedback indicative of completion of the step; and verifying correctness of the list before providing a subsequent step of the FOS to the workstation.

2. The method of claim 1, wherein the feedback includes an image or a computed state of an object affected by the step and wherein verifying correctness of the step comprises employing artificial intelligence-based visual inspection and computation of the image to determine whether or not the step was performed correctly.

3. The method of claim 1, wherein the instructions for manufacturing include instructions for one or more associated processes, including assembly, disassembly, rework, testing, maintenance, field repair, safety, calibration, installation, fault isolation, removal, replacement, packing, unpacking, and training.

4. The method of claim 1, wherein at least one instruction of at least one FOS is performed by a robotic tool of at least one of the workstations.

5. The method of claim 1, wherein the FOS include one or more of: text, images, video, 2D models, 3D models, and worksheets.

6. The method of claim 1, wherein a user interface is configured for access by one or more people, each assigned one or more roles of:(i) a software administrator for installing the software;(ii) an organization administrator for managing access control services, including assignment of projects and roles, to other users;(iii) a project manager for supervising production project knowledge base creation, completeness, permissions and approvals;(iv) a research and development (R&D) engineer at an early stage of product design, as well as when updating a product with new features and creating new product versions, when redesigning due to malfunctions, and when building a new product that has common features with a previous one;(v) an NPI engineer for managing the whole process of new product introduction flow process, communications, digital engineering production portfolio and documentation between the R&D, engineering and production departments;(vi) an engineer responsible for converting a prototype design to a production design, for verifying the completeness of a design model and requesting and / or adding missing information, creating the additional digital engineering production portfolio and documentation;(vii) an internal or external documenter responsible for creating and completing the engineering digital engineering production portfolio and documentation; and,(viii) a manager for viewing and approving the creation, completeness, permissions and approvals of the project production knowledge base and engineering documentation.

7. The method of claim 1, wherein said interactive production portfolio is represented by a structured engineering procedural visual language enforcing standardization and having procedures, steps, and actions, and wherein the interactive production guide includes options for:(i) leading a user through self-guided flow driven procedure creation;(ii) leading a user through a creation validation process including one or more of machine vision, similarity algorithms, rules-based algorithms, semantic context-based generation, and deep learning algorithms;(iii) allowing a direct manipulation free procedure workflow creation; and(iv) allowing a top-down, bottom- up, or singleton creation process.

8. The method of claim 1, wherein the interactive production portfolio includes a definition of an interface prompting a user to choose a procedure type to be generated and guiding him through a procedure creation process.

9. The method of claim 1, wherein the interactive production portfolio includes triggering events for procedures execution, such as: time events, fault event etc.

10. The method of claim 1, wherein a training procedure generates a visual display of steps to be learned by a user, wherein the training procedure is a single step or a FOS, and wherein training presented by the training procedure provides certification for a specific task, or “refreshing” mandated by regulatory agents, or real-time “help” assistance to a user.

11. A system of scheduling and issuing interactive instructions for manufacturing of a product and product sub-assemblies, implemented by one or more processors having nontransient memory including instructions that perform steps comprising: accessing a project production engineering knowledge base including an interactive production portfolio including a flow of steps (FOS) defining sequences of instructions for manufacturing the product and sub-assemblies, and including one or more required resource definitions of tools and materials required at each of multiple workstations at which the FOS are performed, and further including specification of workstation assignments, assigning the FOS and definitions to each of multiple workstations; receiving a workplan specifying target times for completing one or more of the product and sub-assemblies; generating, according to the product portfolio, the workstation assignments, and the workplan, a schedule for performing the assigned FOS of each workstation, wherein the FOS schedule for each workstation is generated to meet FOS requirements of related tasks at other workstations; andproviding to each workstation a first step of the assigned FOS at the scheduled time; receiving from the workstation, during each step of the FOS implemented by the workstation, a list indicating the actions performed, tools, materials, and jigs used, and feedback indicative of completion of the step; and verifying correctness of the list before providing a subsequent step of the FOS to the workstation.

Images