Method, non-transitory computer readable medium and system for monitoring a manufacturing process - Patents.com

Abstract

To provide a method for monitoring a manufacture process, and a non transitory computer readable medium and system which provide a resolution and recommendation concerning manufacture monitoring, troubleshooting, or analysis, and facilitate continuous improvement activities.SOLUTION: The present invention is a method for monitoring a manufacture process including acquiring manufacture data associated with a manufacture machine employed in a manufacture process, acquiring operator data associated with an operator who works the manufacture machine, acquiring material data associated with a material employed by at least one of the operator or the manufacture machine of the manufacture process, calculating a calculation dataset on the basis of the manufacture data, operator data, and material data, and performing analysis at least on the operator data on the basis of the resultant calculation dataset.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a method, a non-transitory computer readable medium and a system thereof for monitoring a manufacturing process. [Background technology]

[0002] Digital transformation has been and will continue to be one of the most important topics across various industries, especially manufacturing. Different technologies and tools have emerged around the application of the Internet of Things (IoT) for industrial applications, such as smart manufacturing and the Industrial Internet of Things (IIoT). Digital solutions using big data paint a great picture for the future of manufacturing, i.e. visualized and quantified manufacturing processes, centralized information analysis and decision-making, remote / cyber-physical manufacturing systems, which further lead to operational cost reduction, improved efficiency at all levels, and revenue growth. As a result, manufacturers, regardless of their company size and current digitalization stage, are riding this wave of digital transformation.

[0003] Unfortunately, companies may experience problems using digital solutions and are stuck with traditional information management systems such as whiteboards, paper documents, or spreadsheets. This may be caused by many factors, such as but not limited to a lack of digital data acquisition and analysis, especially in the case of smaller manufacturers with legacy equipment that cannot easily provide digital data. The IIoT requires several sensors for digital data collection, which may lead to problems in its application, especially in sensor development, selection, installation, integration, and / or data analysis. Patent Documents 1 and 2 disclose the following:

[0004] In one example, the related art describes a system for action recognition to determine cycles, processes, actions, sequences, objects in one or more sensor streams. The system may include analysis tools that may include real-time verification of actions and packing or unpacking by image recognition.

[0005] In one example, the related art describes a system incorporating devices that capture events and measurements recorded by devices and sensors as inputs and outputs for build triggers and time-saving workflows.

[0006] In one example, the related art describes a monitoring system that monitors people, objects, or devices in a manufacturing location, enabling accurate understanding of people and object movements in the manufacturing process to improve work efficiency.

[0007] In one example, the related art describes a system for automatically recording work content and work time performed by multiple devices, such that a work analysis device extracts content of events generated by devices in the vicinity of an operator within a time segment and generates work analysis information by associating the extracted event content with the time segment.

[0008] In one example, the related art describes a system that analyzes a combination of manufacturing results and manufacturing plans and extracts elements that are candidates for improvement. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] JP 2003-140728 A [Patent Document 2] Patent Publication No. 2021-196909 Summary of the Invention [Problem to be solved by the invention]

[0010] Although the above-mentioned patents and patent literature 2 may seek to improve the manufacturing process, one of the major problems in collecting operational data is that most of the manufacturing processes in the manufacturing industry involve different levels of manual operations. However, operational data involving manual operations is still difficult to track as digitized data that can be analyzed, let alone real-time data. Another problem is that the above examples may use camera / video-based techniques that may lead to the acceptance of operators on the production floor, along with privacy and legal concerns, which may be undesirable for the operation of the production floor.

[0011] The present invention involves monitoring the operational status of a multi-sensor system. Thus, the present invention provides a system for monitoring the operational status of a multi-sensor system by utilizing a non-vision based multi-sensor system for collection of human and other data. The system may apply advanced artificial intelligence (AI) / machine learning (ML) techniques with sensor fusion for comprehensive operational data analysis. Machine data and status may be monitored through automation and digitization. At the same time, real-time operator status data is important for manufacturers to understand ongoing operations including manufacturing efficiency, quality, throughput, safety, etc.

[0012] A problem with related art is that there have been issues with obtaining real-time operator status, such as manufacturers being unable to optimize / maximize resource (e.g., machine and worker) utilization due to lack of real-time operator status (e.g., availability, programming, installing tools, running machines, etc.). For example, managers typically only have information about the operator's shift times (records of clock-in and clock-out times) versus the actual operations performed by the operator and their duration. A typical problem is that machine utilization does not match operator utilization, i.e., the machine is running and the operator is working (ideal case), the machine is running and the operator is away (waste of human man-hours), the machine is idling and the operator is away (waste of both machine and human man-hours).

[0013] Another problem with related techniques is that manufacturers are unable to identify operational bottlenecks for processes with many manual steps when machine and / or human utilization is low. Therefore, due to the inability to quantify the operator time spent in each work zone / job step (e.g., cycle time), resources cannot be allocated to address existing line bottlenecks, allocate training for underperforming operators, or provide insights to improve future line design (e.g., how many operators per task). Cycle times for manual steps can be obtained to provide quantified data for bottleneck identification / analysis.

[0014] Another problem with related techniques is that the walking path or motion trajectory of the operator is often not recorded / tracked, so optimization of the physical layout cannot be performed. The cycle time of such activities cannot be tracked or analyzed. For example, an operator needs to walk to multiple locations to get different parts, and an efficient layout needs to be determined.

[0015] Another problem with related techniques is quality issues and warranty claims caused by the inability to identify non-standardized work procedures (e.g., an operator skipping / missing a step, an operator installing the wrong part, performing in the wrong order), and historical data for each manufacturing step can provide traceability for quality events such as recalls, warranty issues, etc.

[0016] Another problem with related art is that standardized work instructions are often pre-programmed and timed sequences. Sensor data from operator actions can be used to trigger work instructions to aid in the flow of the manufacturing process. For example, an operator may finish a step, a sensor may send a signal indicating completion of the step, and work instructions for the subsequent step may be presented to the operator.

[0017] The object of the present invention is to provide solutions and recommendations for production monitoring, troubleshooting, and analysis, Prevent material waste by specific operators, It is a further object of the present invention to provide a method, a non-transitory computer readable medium and a system thereof for monitoring a manufacturing process that promotes continuous improvement activities. [Means for solving the problem]

[0018] The present invention relates to a method for monitoring a manufacturing process, the method comprising: acquiring production data corresponding to a manufacturing machine utilized in the manufacturing process; acquiring operator data corresponding to an operator operating the manufacturing machine; acquiring material data corresponding to materials being used by at least one of the operator or the manufacturing machine in the manufacturing process, the material data including a material usage rate; and calculating a calculation data set based on the production data, the operator data, and the material data; Analyzing the 4Ms (human, machine, material and method data) to prevent material waste by specific operators and performing an analysis on at least the operator data based on results of the computational data set.

[0019] Alternatively, the invention is directed to a non-transitory computer readable medium storing instructions for execution by one or more hardware processors for monitoring a manufacturing process, the instructions including: acquiring manufacturing data corresponding to a manufacturing machine utilized in the manufacturing process; acquiring operator data corresponding to an operator operating the manufacturing machine; acquiring material data corresponding to materials being used by at least one of the operator or the manufacturing machine in the manufacturing process, the material data including a material usage rate; and calculating a computational data set based on the manufacturing data, the operator data, and the material data. Analyzing the 4Ms (human, machine, material and method data) to prevent material waste by specific operators and performing an analysis on at least the operator data based on results of the computational data set.

[0020] Alternatively, the present invention may include a manufacturing machine utilized in a manufacturing process; and a processor configured to: acquire manufacturing data corresponding to the manufacturing machine; acquire operator data corresponding to an operator operating the manufacturing machine; acquire material data corresponding to materials used by at least one of the operator or the manufacturing machine in the manufacturing process, the material data including a material usage rate; and calculate a calculation data set based on the manufacturing data, the operator data, and the material data; Analyzing the 4Ms (human, machine, material and method data) to prevent material waste by specific operators and performing an analysis on at least the operator data based on results of the computational data set. Effect of the Invention

[0021] In accordance with the present invention, solutions and recommendations for production monitoring, troubleshooting, and analysis are provided; Prevent material waste by specific operators, Additionally, a method, non-transitory computer readable medium, and system for monitoring a manufacturing process may be provided that facilitates continuous improvement activities. [Brief description of the drawings]

[0022] [Figure 1] FIG. 2 illustrates an example system architecture of the system. [Figure 2A] FIG. 2 illustrates the process functions and features of the system. [Figure 2B] FIG. 2 illustrates the process functions and features of the system. [Figure 2C] FIG. 2 illustrates the process functions and features of the system. [Figure 3A] FIG. 1 illustrates an example of a human data analysis process. [Figure 3B] FIG. 1 illustrates an example of a human data analysis process. [Figure 4A] FIG. 1 illustrates an example of a multi-sensor system. [Figure 4B] FIG. 1 illustrates an example of a multi-sensor system. [Diagram 5] FIG. 13 is a diagram showing an example of how an operator uses a sensor. [Figure 6] FIG. 1 illustrates an example of tracking operator movements. [Figure 7] FIG. 1 illustrates an example chart and UI. [Figure 8] FIG. 13 illustrates an example of using a sensor signal for active work order prompts. [Figure 9] FIG. 13 is a diagram illustrating an example of cycle time improvement. [Figure 10] FIG. 13 illustrates an example of material savings. [Figure 11] FIG. 13 illustrates an example of method optimization. [Figure 12] FIG. 13 is a diagram illustrating an example of bottleneck identification and countermeasures. [Figure 13] FIG. 13 illustrates an example of operation monitoring. [Figure 14] FIG. 1 illustrates an example of motion path monitoring and refinement. [Figure 15] FIG. 1 illustrates an example of quality monitoring and improvement. [Figure 16]FIG. 1 illustrates an example of an automated work order prompt. [Figure 17] FIG. 1 illustrates an exemplary computing environment having an exemplary computing device suitable for use in some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] To address this problem in the related art, the embodiments described herein involve the following aspects.

[0024] The embodiments described herein involve novel techniques for monitoring the operational status of a multi-sensor system. The system may comprise a non-visual based multi-sensor system having sensors deployed to monitor the presence and activity of operators at various key locations in a manufacturing environment. The sensor system may provide real-time digitized manufacturing related information about the operators. Such data may be combined with other manufacturing data (e.g., machine status, job scheduling, part tracking, quality, etc.) and digital services are provided to analyze such data and provide solutions and recommendations for manufacturing monitoring, troubleshooting, analysis, and to help drive continuous improvement activities.

[0025] In an embodiment involving this system, the system is configured to monitor a manufacturing process, and may include at least one or more of a sensor system that acquires data from a manufacturing machine, a sensor system that acquires data from an operator, a sensor system that acquires data from materials used in the manufacturing process, a calculation unit that performs calculations based on the data acquired from the sensor, an analysis unit that uses AI to analyze at least the data acquired from the operator based on the result of the calculation, or a display unit that notifies the operator of the analysis result and countermeasures.

[0026] Aspects of the disclosure include a method that involves obtaining manufacturing data corresponding to a manufacturing machine utilized during a manufacturing process, obtaining operator data corresponding to an operator operating the manufacturing machine, obtaining material data corresponding to materials used by at least one of the operator or the manufacturing machine in the manufacturing process, calculating a computational data set based on the manufacturing data, the operator data, and the material data, and performing an analysis on at least the operator data based on results of the computational data set.

[0027] Aspects of the present disclosure further include a computer program storing instructions involving obtaining manufacturing data corresponding to a manufacturing machine utilized during the manufacturing process, obtaining operator data corresponding to an operator operating the manufacturing machine, obtaining material data corresponding to materials used by at least one of the operator or the manufacturing machine in the manufacturing process, calculating a computational data set based on the manufacturing data, the operator data, and the material data, and performing an analysis on at least the operator data based on results of the computational data set.

[0028] Aspects of the present disclosure include a system with means for acquiring manufacturing data corresponding to a manufacturing machine utilized during a manufacturing process, means for acquiring operator data corresponding to an operator operating the manufacturing machine, means for acquiring material data corresponding to materials used by at least one of the operator or the manufacturing machine in the manufacturing process, means for calculating a computational data set based on the manufacturing data, the operator data, and the material data, and means for performing an analysis on at least the operator data based on results of the computational data set.

[0029] An aspect of the disclosure involves an apparatus that facilitates monitoring a manufacturing process, the apparatus including a processor configured to: acquire manufacturing data corresponding to a manufacturing machine utilized during the manufacturing process; acquire operator data corresponding to an operator operating the manufacturing machine; acquire material data corresponding to a material used by at least one of the operator or the manufacturing machine in the manufacturing process; calculate a computational data set based on the manufacturing data, the operator data, and the material data; and perform an analysis on at least the operator data based on results of the computational data set. EXAMPLES

[0030] The following detailed description provides details of the figures and examples of the present application. Reference numbers and descriptions of elements that overlap between figures are omitted for clarity. Terms used throughout the description are provided as examples and are not intended to be limiting. For example, the use of the word "automatic" may include fully automated or semi-automatic implementations, including user or administrator control over certain aspects of the embodiment, depending on the desired implementation of the person skilled in the art practicing the embodiments of the present application. Selections may be performed by a user via a user interface or other input means, or may be implemented via a desired algorithm. Examples as described herein may be utilized either alone or in combination, and the functionality of the examples may be implemented via any means according to the desired implementation.

[0031] FIG. 1 shows an example of an overall system architecture for system 100. The system may include data acquisition (e.g., 102, 104), calculation, analysis, storage, and display 106. In some embodiments, the collected data may include multi-sensor systems and raw data, for example, as shown at 102 in FIG. 1. In some embodiments, the collected data may include other data included directly from machines, job databases, material databases, etc., for example, as shown at 104 in FIG. 1. The collected data may be processed by a server and analyzed by system 100 such that results are output to display 106.

[0032] 2A-2C show detailed functional and feature diagrams of the process that may be performed by the system 100 shown in FIG. 1. Although the system can be used to collect any of human, machine, material, and method data (4M), the advantage of the system is its ability to collect digitized and quantified data related to humans and / or operators. The system takes in raw data from the multi-sensor system and other relevant manufacturing data such as the physical layout of the environment, job information, schedule / work instructions, etc., and then performs pre-processing of the human / operator data based on custom-built algorithms. The analyzed human data may then be combined with other similarly acquired data such as, but not limited to, machine, method, or material data for further advanced 4M analysis. The final analysis output is a higher level data that directly represents the manufacturing status through AI / ML-based advanced analysis, which may further provide one or more recommendations for continuous improvement, bottleneck analysis, solution, etc. Additionally, the UI may present the analyzed information along with feedback and recommendations for the end user.

[0033] In some embodiments, raw human data (e.g., multi-sensor systems) may include time-stamped sensor trigger data, sensor measurement data (e.g., distance, proximity, acoustic signal, etc.), sensor configuration data, sensor location (e.g., x,y,z position), sensing range, trigger conditions, etc., or sensor configuration parameters (e.g., sensitivity, sampling frequency, output frequency, or type). In some embodiments, raw machine data (e.g., machine connections and / or multi-sensor systems) may include time-stamped sensor trigger data, sensor measurement data (e.g., distance, proximity, acoustic signal, etc.), sensor configuration data, sensor location (e.g., x,y,z position), sensing range, trigger conditions, etc., sensor configuration parameters (e.g., sensitivity, sampling frequency, output frequency, or type), or direct machine data. In some embodiments, raw human data (e.g., multi-sensor system and other data) may include time-stamped sensor trigger data, sensor measurement data (e.g., distance, proximity, acoustic signals, etc.), sensor configuration data, sensor location (e.g., x, y, z position), sensing range, trigger conditions, etc., sensor configuration parameters (e.g., sensitivity, sampling frequency, output frequency, or type), or other method data (e.g., procedure scheduling, job details, designs).

[0034] In some embodiments, human data analysis may include combining and storing all raw data in a database, combining raw sensor data, applying analytical models for analysis and simplification, outputting human / operator operation status, or applying AI / ML on the raw data and utilizing advanced analytics to provide accurate human data. In some embodiments, the analyzed human data may include human / operator time-stamped operation status, timestamp and duration of each manual operation step (e.g., cycle time), human / operator walking path / motion trajectory, completion status of each manual operation step, or quality of manual operation steps.

[0035] In some embodiments, machine data analysis may include combining and storing all raw data in a database, combining raw sensor data, applying analytical models to analyze and simplify, outputting machine operational status, or applying AI / ML on the raw data and utilizing advanced analytics to provide accurate machine data. In some embodiments, the analyzed machine data may include time-stamped operational status of the machine, machine cycle time / run time, other process data related to the machine (e.g., run time, performance, detailed parameters), or machine job quality.

[0036] In some embodiments, process data analysis may include combining and storing all raw data in a database, combining raw sensor data, applying analytical models to analyze and simplify, outputting process operation status, or applying AI / ML on the raw data and utilizing advanced analytics to provide accurate process data. In some embodiments, the analyzed process data may include analysis of actual procedure data compared to standardized work procedures, scheduling information, job related details (e.g., work orders, target cycle times, target production volumes), shift information, work hours, operation schedules, or job lead times.

[0037] In some embodiments, material data analysis may include combining and storing all raw data in a database, combining raw sensor data, applying analytical models to analyze and simplify, outputting the operational status of the material, or applying AI / ML on the raw data and utilizing advanced analytics to provide accurate method data. In some embodiments, the analyzed material data may include material related information (e.g., design, quality, dimensions, quantity), actual sensor measurements related to the material (e.g., material, dimensions, quality, quantity, design), or other information.

[0038] In some embodiments, 4M analytics may include combining and analyzing 4M data using advanced technologies (e.g., AI / ML) or analyzing manufacturing / operational data to output overall operational status and performance. In some embodiments, 4M analytics may include human and machine utilization, real-time 4M status, manufacturing status and historical analysis (e.g., efficiency, throughput, quality, progress), productivity loss in manufacturing quantified by time and location, detailed productivity loss information and root cause based on each 4M component (e.g., operator absent, machine down, tool broken), quality information (e.g., completion status or quality checkpoint inspection results), real-time manufacturing information on schedule, or AI / ML-based recommendations for continuous improvement or solutions to manufacturing problems.

[0039] In some embodiments, the visualization or display of results (e.g., user interface (UI)) may include real-time 4M status, as well as a visualization or UI for displaying all 4M information and analysis results, live plotting, interactive tools for data selection, calculations, analysis, communication methods (e.g., messaging between production site members and management, posts for leaving instructions or notes).

[0040] 3A-3B show an example of a human data analysis process. In some embodiments, raw data from various sensors may be collected, filtered, and analyzed using an advanced AI / ML model 302. The model may be pre-trained or configured for real-time training and reinforcement. Through such analysis, direct operation data 304 may be obtained. Operation data 304 may include information related to real-time status, cycle time tracking, walk path tracking, or operation sequence. Real-time status may provide information related to a current step or stage of a process. For example, real-time status may indicate that a manual operation step is being performed or has been performed at a given time. Examples may include real-time operator status, manual cycle time tracking, manual operation sequence, completion status of each manual operation, direct quality information from sensor measurements, and operator walk path tracking.

[0041] Some of the problems to be solved using AI / ML analytics may include determining operational actions when several sensors are triggered simultaneously (e.g., linking multiple sensor data for complex problems / actions), identifying errors in operator actions when sensor readings are fluctuating normally (e.g., noise), and modeling longer ranges of sensor data (e.g., weeks, months) to pinpoint systemic issues such as wear on machinery, slowdowns, or poor operator performance.

[0042] 4A and 4B show example configurations of multi-sensor systems 400, 410 having sensors deployed to monitor operator presence and activity at various key locations in a manufacturing environment (e.g., a production line and computer numerically controlled machining). The sensors may include any of acoustic, ultrasonic, photoelectric, infrared, or any other non-visual based sensors.

[0043] Sensor detection may be triggered by operator presence and / or action to indicate a change in status or measurement data, such as, but not limited to, distance, shape, reflectance, temperature, etc., that may be converted into operation status related data. In some embodiments, operator presence or action may be monitored by sensors covering each platform (e.g., Platform 1, Platform 2, or Platform 3) respectively. In some embodiments, an operator may perform an operation on a work platform, which is triggered either by the operator's presence in front of the work platform or by detailed hand movements on the work platform surface. Similarly, motion may be detected either by the operator's presence in front of a shelf, or hand movements on a shelf, or action when lifting a part. Areas that may be monitored may also include machines / work platforms, human-machine interaction systems (HMI), computer desks, etc.

[0044] 5 illustrates an example of using sensors by an operator 506. These sensors may be configured to identify the operator's actions. For example, detection of the operator's hand movement may be indicative of an action by any one of sensors 502 or 504. Similarly, sequences of such actions may also be tracked and recorded. As shown in FIG. 5, sensor 502 may detect the presence of hand 508 of operator 506 such that the operator's hand is not within the vicinity of sensor 502 such that the operator's hand movement when performing a task may be detected by sensor 504.

[0045] FIG. 6 shows an example diagram 600 of an operator's walking path or motion trajectory tracking. Such information can be used for cycle time tracking and also for checking against standardized work procedures. Furthermore, the data can be used for optimization of the physical layout of the workspace and / or the standardized work procedures to improve the efficiency of the work environment and procedures. Similarly, the motion trajectory is not limited to the operator's position, but can be the trajectory of the operator's hands or other moving parts. In the example of FIG. 6, the numbered arrows can show a step-by-step process that an operator may perform while working. For example, an operator may start at an HMI, then go to a machine at 1, then go to a workbench at 2, then go to a shelf at 3, then go to a workbench at 4, then go to a personal computer (PC) at 5, then go to a machine at 6. In some embodiments, the machine may include one or more conveyors, forklifts, workspace, or any other manufacturing system / device that may be used in a manufacturing process. While this is an example process that may be performed to perform a task, the present disclosure is not intended to be limited to the example shown in FIG. 6. In some aspects, the operator may perform different steps in a different order or may not need to visit or use all or some components of the workspace.

[0046] 7 shows a diagram 700 of an example chart and UI that can be utilized by an end user to provide a visual representation of data. Example visualizations can include one or more machines, operation time usage charts, operation-by-operation completion tracking and cycle time tracking, operator walking path / trajectory tracking, job / operation progress tracking, and the like, or combinations thereof.

[0047] Figure 8 shows a schematic diagram 800 of an example of using sensor signals for active work instruction prompts during operation. As shown in 802 of Figure 8, a current visual instruction is shown for step 3. When the sensor detects the completion of the operation of step 3, the signal can be used to trigger the work instruction for the next step (e.g., step 4), as shown in 804 of Figure 8. The sensor signal can be used for different modalities of work instructions, such as, but not limited to, on-screen display, visual projection images, sound effects, voice instructions, etc.

[0048] FIG. 9 shows a diagram of cycle time improvement 902. For example, cycle time improvement may include instances where an increase in cycle time may occur to adjust the overall manufacturing process. For example, if the system detects that an operator is making a mistake over time, the system checks the machine status to determine the root cause of the mistake. In some embodiments, if the system detects a machine failure, the system may gain insight from the machine data and further, human data may be drawn upon to determine the problem with the machine. After the data is analyzed, the system may determine if there is a way to address the root cause with the machine. The system may generate recommendations based on historical data and / or knowledge. The system may provide instructions on how to solve the problem with the machine. For example, the UI may include suggestions and instructions provided based on historical data, experience, manual knowledge, etc. At least one advantage is to provide improvements that include cycle time analysis and human data. Conventional methods typically use only machine data without using cameras to collect human data.

[0049] FIG. 10 shows a diagram of material savings 1002. For example, the system may obtain data on actual material usage against job description / instructions to reference material usage for irregular data points and identify related potential causes. In some examples, certain operators may waste more during production than other operators. To prevent material waste by certain operators, the system may take some countermeasures. For example, some countermeasures may be based on real-time data and AI analysis, and recommendations may be provided to the operators for remedial actions. Also, notifications may be sent to supervisors to further investigate the issue in detail. The system may improve traditional data and analysis with accurate human operation data. Direct data from operators provides further details of root cause issues and corresponding suggestions.

[0050] FIG. 11 shows a diagram of process optimization 1102. For example, the system may look at the scheduled upcoming jobs for the day and check the status of people, machines, materials along with different process data that are used to find the best combination and solution. The system may recommend optimal execution schemes for operation schedules, machine schedules, parts / order routing schedules. The system can achieve improved data analysis with the addition of human data and AI models can better suggest countermeasures and / or improvements.

[0051] FIG. 12 shows an illustration of bottleneck identification and countermeasures 1202. For example, the system may track cycle times and identify steps that are taking longer than expected cycle times and the root cause (e.g., setup time or preparation on the machine). The system may analyze 4M data over a longer time period to identify overall issues with manufacturing bottlenecks. In some examples, there may be a variety of root causes including, but not limited to, method issues in product or process design, machine issues with degraded machine performance due to wear / lack of maintenance, human issues with not enough human resources to complete the job on time (also a method issue) or lack of skills to complete on time, etc. The system may recommend actions for improvement (e.g., additional training, improved work instructions). The system can provide more accurate bottleneck identification and further details with granular information directly from the actions by the operators, and more detailed data and analysis also leads to more accurate and effective solution recommendations.

[0052] 13 shows a diagram of operational monitoring 1302. For example, the system may monitor status such as throughput, utilization, downtime, and quality. The system may present the monitored data and provide operational data of the production floor. The system may provide further visibility into the operational data of the production floor along with detailed human action data.

[0053] FIG. 14 shows a diagram of walk / motion path monitoring and improvement 1402. For example, the system may perform walk / motion path monitoring of operators. For example, the system may compare human data to standardized / optimal trajectories. The results can be used for quality tracking or cycle time calculation. Walk / motion paths can be linked with throughput, quality, and operation schemes (e.g., methods) to provide another level of information for shop floor visibility, such as, but not limited to, motion paths to identify missing steps that may cause quality issues, monitor paths to calculate cycle times, or check used path data to identify optimal schemes. The system may provide another perspective of operations that can serve as supplemental data for cycle time tracking, quality measurement, and schedule optimization.

[0054] FIG. 15 shows a diagram of quality monitoring and improvement 1502. The system may perform quality tracking, which may include real-time inspection and quality checks related to manual operations and part quality (e.g., verifying that all steps during assembly are performed and completed, all required parts are installed, and proper placement of parts is performed). The system may track the completion and quality status of each step in the manufacturing scheme to ensure proper execution of all required steps. The system may provide real-time inspection and reporting of quality information, as well as store all quality data in a database for traceability. The system may enable real-time quality inspection and accounting during some preliminary rapid processes.

[0055] 16 shows a diagram of an automated work order prompt 1602. The system can provide dynamic step-by-step work instructions based on current operator actions (e.g., completion of a current step). The system can dynamically and interactively provide work instructions, thereby providing the operator with step-by-step pictorial instructions for longer jobs. The system can provide an effective way to provide on-demand work instructions on the production floor, especially for operations with many steps.

[0056] Additionally, the system can be used either as a stand-alone software or as an add-on module as a data collection and analysis tool to existing systems such as, but not limited to, Supervisory Control and Data Acquisition (SCADA) or Manufacturing Execution Systems (MES).

[0057] The system provides an easy way to monitor and collect operational data from many manual operations, which may solve the problems described herein. The system may operate as a standalone system, allowing it to be installed in an existing manufacturing environment with minimal impact on current operations and configurations. The sensors selected may be highly versatile and adaptable to many operating modalities. The system is easily scalable with the addition of sensors and data hubs. At least one advantage of the present disclosure is that non-visual based data collection is available for this system, making it more appealing to most customers and live operators.

[0058] FIG. 17 illustrates an exemplary computing environment having an exemplary computing device suitable for use in some embodiments, as illustrated in connection with any of FIGS. 1-16. The computing environment can be used to facilitate implementation of the architecture illustrated in FIGS. 1-16. Additionally, any of the embodiments described herein can be implemented based on the architecture, API, microservices system, etc., as illustrated in FIGS. 1-16. The computing device 1705 in the computing environment 1700 can include one or more processing units, cores, or processors 1710, memory 1715 (e.g., RAM, ROM, and / or the like), internal storage 1720 (e.g., magnetic, optical, solid-state storage, and / or organic), and / or I / O interface 1725, any of which can be coupled over a communication mechanism or bus 1730 for communicating information or can be incorporated into the computing device 1705. The I / O interface 1725 is further configured to receive images from a camera or provide images to a projector or display, depending on the desired implementation.

[0059] The computing device 1705 may be communicatively coupled to an input / user interface 1735 and an output device / interface 1740. Either one or both of the input / user interface 1735 and the output device / interface 1740 may be a wired or wireless interface and may be removable. The input / user interface 1735 may include any device, component, sensor, or interface, physical or virtual, that can be used to provide input (e.g., buttons, touch screen interface, keyboard, pointing / cursor control, microphone, camera, Braille, motion sensor, optical reader, and / or the like). The output device / interface 1740 may include a display, television, monitor, printer, speaker, Braille, or the like. In some examples, the input / user interface 1735 and the output device / interface 1740 may be incorporated with the computing device 1705 or may be physically coupled thereto. In other embodiments, other computing devices may function as or provide the functionality of input / user interface 1735 and output device / interface 1740 for computing device 1705 .

[0060] Examples of computing devices 1705 may include, but are not limited to, highly mobile devices (e.g., smart phones, devices mounted on vehicles and other machines, devices carried by people or animals, and the like), mobile devices (e.g., tablets, notebooks, laptops, personal computers, portable televisions, radios, and the like), and devices not designed for mobility (e.g., desktop computers, other computers, information kiosks, televisions with one or more processors embedded and / or coupled to one or more processors, televisions, radios, and the like).

[0061] Computing device 1705 may be communicatively coupled (e.g., via I / O interface 1725) to external storage 1745 and a network 1750 for communication with any number of networked components, devices, and systems, including one or more computing devices of the same or different configurations. Computing device 1705 or any connected computing device may function as, provide services to, or be referred to as a server, client, thin server, general purpose machine, special purpose machine, or otherwise.

[0062] I / O interface 1725 may include, but is not limited to, wired and / or wireless interfaces using any communication or I / O protocol or convention (e.g., Ethernet, 802.11x, Universal system Bus, WiMax, modem, cellular network protocols, and the like) for communicating information to and / or from at least all connected components, devices, and networks in computing environment 1700. Network 1750 may be any network or combination of networks (e.g., the Internet, a local area network, a wide area network, a telephone network, a cellular network, a satellite network, and the like).

[0063] The computing device 1705 can use and / or communicate using computer usable or computer readable media, including transitory and non-transitory media. Transitory media includes transmission media (e.g., metallic cables, optical fibers), signals, carrier waves, and the like. Non-transitory media includes magnetic media (e.g., disks and tapes), optical media (e.g., CD ROM, digital video disks, Blu-ray DISC), solid-state media (e.g., RAM, ROM, flash memory, solid-state storage), and other non-volatile storage or memory. Blu-ray DISC is a registered trademark of the Blu-ray Disc Association.

[0064] The computing device 1705 may be used to implement techniques, methods, applications, processes, or computer-executable instructions in some exemplary computing environments. The computer-executable instructions may be retrieved from a transitory medium and stored in and retrieved from a non-transitory medium. The executable instructions may be from one or more of any programming language, scripting language, and machine language (e.g., C, C++, C#, Java, Visual Basic, Python, Perl, JavaScript, and others).

[0065] The processor 1710 can run under any operating system (OS) (not shown) in a native or virtual environment. One or more applications can be deployed, including a logic unit 1760, an application programming interface (API) unit 1765, an input unit 1770, an output unit 1775, and an inter-unit communication mechanism 1795 for different units to communicate with each other, with the OS, and with other applications (not shown). The above-mentioned units and elements can differ in design, function, configuration, or implementation, and are not limited to the above description. The processor 1710 can have the form of a hardware processor, such as a central processing unit (CPU), or can be a combination of hardware and software units.

[0066] In some examples, when information or instructions for execution are received by the API unit 1765, it may be communicated to one or more other units (e.g., logic unit 1760, input unit 1770, output unit 1775). In some examples, logic unit 1760 may be configured to control information flow between units and direct services provided by API unit 1725, input unit 1770, output unit 1775 in some examples described above. For example, one or more process or implementation flows may be controlled by logic unit 1760 alone or in conjunction with API unit 1765. Input unit 1770 may be configured to obtain inputs for calculations described in the examples, and output unit 1775 may be configured to provide outputs based on the calculations described in the examples.

[0067] The processor 1710 can be configured to execute instructions for a method, for example in any of Figures 1-16, involving obtaining manufacturing data corresponding to a manufacturing machine utilized during a manufacturing process, obtaining operator data corresponding to an operator operating the manufacturing machine, obtaining material data corresponding to materials used by at least one of the operator or the manufacturing machine in the manufacturing process, calculating a computational data set based on the manufacturing data, the operator data, and the material data, and performing an analysis on at least the operator data based on results of the computational data set.

[0068] The processor 1710 can be configured to execute instructions for a method, for example in any of Figures 1-16, which provides as output a user interface including results of analysis of at least the operator data based on the results of the computational data set.

[0069] Some portions of the detailed description are presented in terms of algorithms and symbolic representations of operations within a computer. These algorithmic descriptions and symbolic representations are the means used by those skilled in the data processing arts to convey the substance of their innovations to others skilled in the art. An algorithm is a sequence of defined steps leading to a desired end state or result. In the examples, the steps performed require physical manipulations of tangible quantities to achieve a tangible result.

[0070] Unless otherwise noted, and as will be apparent from the description, throughout this specification, descriptions utilizing words such as "processing," "computing," "calculating," "determining," "displaying," or the like, are understood to include the actions and processes of a computer system or other information processing device that manipulates and converts data represented as physical (electronic) quantities in the registers and memory of the computer system into other data similarly represented as physical quantities in the memory or registers of the computer system or other information storage, transmission, or display devices.

[0071] Examples may further relate to an apparatus for performing the operations herein. The apparatus may be specially constructed for the desired purposes or may include one or more general-purpose computers selectively activated or reconfigured by one or more computer programs. Such computer programs may be stored in a computer-readable medium, such as a computer-readable storage medium or a computer-readable signal medium. A computer-readable storage medium may include tangible media, such as, but not limited to, optical disks, magnetic disks, read-only memory, random access memory, solid-state devices and drives, or any other type of tangible or non-transitory medium suitable for storing electronic information. A computer-readable signal medium may include media such as carrier waves. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. A computer program may include a pure software implementation that includes instructions for performing the operations of a desired embodiment.

[0072] Various general-purpose systems may be used with the programs and modules according to the examples herein, or it may be convenient to construct specialized apparatus to perform the steps of the desired method. Moreover, the examples are not described with reference to any particular programming language. It will be understood that various programming languages ​​may be used to implement the techniques of the examples as described herein. Instructions of the programming language may be executed by one or more processing devices, such as, for example, a central processing unit (CPU), a processor, or a controller.

[0073] As is known in the art, the operations described above may be performed by hardware, software, or some combination of software and hardware. Various aspects of the embodiments may be implemented using circuits and logic devices (hardware), while other aspects may be implemented using instructions stored on a machine-readable medium (e.g., software stored in memory 1715) that, when executed by a processor, causes the processor to perform a method to perform an implementation of the present application. Furthermore, some embodiments of the present application may be performed exclusively in hardware, while other embodiments may be performed exclusively in software. Furthermore, the various functions described may be performed in a single unit or may be distributed across multiple components in any number of ways. When performed by software, the method may be executed by a processor, such as a general purpose computer, based on instructions stored on a computer-readable medium. If necessary, the instructions may be stored on the medium in compressed and / or encrypted format.

[0074] Moreover, other implementations of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the present techniques. Various aspects and / or components of the described embodiments may be used alone or in any combination. It is intended that the specification and embodiments be considered as examples only, with a true scope and spirit of the present application being indicated by the following claims. [Explanation of symbols]

[0075] 1735 Input / User Interface 1740 Output Device / Interface 1745 External Storage 1750 Network 1705 Computer Devices 1725 I / O Interface 1720 Internal Storage 1715 Memory 1710 Processor 1760 Logical Units 1770 Input Unit 1765 API unit 1775 Output Unit

Claims

1. 1. A method for monitoring a manufacturing process, comprising: acquiring manufacturing data corresponding to a manufacturing machine utilized in the manufacturing process; acquiring operator data corresponding to an operator operating the manufacturing machine; obtaining material data corresponding to materials being used by at least one of the operator or the manufacturing machine in the manufacturing process, the data including a material usage rate; calculating a calculation data set based on the manufacturing data, the operator data, and the material data; and analyzing 4M, which are man, machine, material and process data, and performing analysis on at least the operator data based on the results of the computational data set to prevent material waste by a particular operator.

2. 2. The method of claim 1, wherein the manufacturing data, the operator data, or the material data is acquired based on a non-vision based sensor system, and at least one of the manufacturing data, the operator data, or the material data is acquired from an existing system or database external or internal to the manufacturing machine.

3. The method of claim 2 , wherein the non-visual based sensor system comprises sensors deployed to monitor the manufacturing machines, the operators, and the materials in the manufacturing process.

4. The method of claim 3 , wherein the sensor monitors for the presence of the operator or activity in the manufacturing process.

5. The method of claim 3 , wherein the non-vision based sensor system provides real-time production information related to the operator, and the computational data set includes the real-time production information.

6. The method of claim 3 , wherein the non-vision based sensor system provides real-time manufacturing information related to the operator, the manufacturing machine, and the materials during one or more stages of the manufacturing process.

7. 7. The method of claim 6, wherein the real-time manufacturing information related to the operators, the manufacturing machines, and the materials during the one or more stages of the manufacturing process is analyzed to provide information related to the one or more stages of the manufacturing process.

8. The method of claim 7 , wherein the information related to the one or more stages of the manufacturing process is configured to troubleshoot or optimize the one or more stages of the manufacturing process.

9. The method of claim 1 , further comprising providing as output a user interface including results of the analysis of at least the operator data based on the results of the computational data set.

10. 1. A non-transitory computer-readable medium storing instructions for execution by one or more hardware processors for monitoring a manufacturing process, the instructions comprising: acquiring manufacturing data corresponding to a manufacturing machine utilized in the manufacturing process; acquiring operator data corresponding to an operator operating the manufacturing machine; obtaining material data corresponding to materials being used by at least one of the operator or the manufacturing machine in the manufacturing process, the data including a material usage rate; calculating a calculation data set based on the manufacturing data, the operator data, and the material data; Analyzing the 4M data of man, machine, material and method, and performing an analysis on at least the operator data based on the results of the computational data set to prevent material waste by a particular operator; A non-transitory computer readable medium comprising:

11. 11. The non-transitory computer-readable medium of claim 10, wherein the manufacturing data, the operator data, or the material data is acquired based on a non-vision based sensor system.

12. 12. The non-transitory computer-readable medium of claim 11, wherein the non-visual based sensor system comprises sensors deployed to monitor the manufacturing machines, the operators, and the materials in the manufacturing process.

13. The non-transitory computer readable medium of claim 12 , wherein the sensor monitors for the presence of the operator or activity within the manufacturing process.

14. 13. The non-transitory computer-readable medium of claim 12, wherein the non-vision based sensor system provides real-time manufacturing information related to the operator, and the computational data set includes the real-time manufacturing information.

15. 13. The non-transitory computer-readable medium of claim 12, wherein the non-visual based sensor system provides real-time manufacturing information related to the operator, the manufacturing machine, and the materials during one or more stages of the manufacturing process.

16. 16. The non-transitory computer readable medium of claim 15, wherein the real-time manufacturing information related to the operators, the manufacturing machines, and the materials during the one or more stages of the manufacturing process is analyzed to provide information related to the one or more stages of the manufacturing process.

17. 20. The non-transitory computer-readable medium of claim 16, wherein the information related to the one or more stages of the manufacturing process is configured to troubleshoot or optimize the one or more stages of the manufacturing process.

18. The non-transitory computer readable medium of claim 10 , wherein the instructions further comprise providing as output a user interface including results of the analysis of at least the operator data based on the results of the computational data set.

19. A manufacturing machine used in the manufacturing process; a processor for obtaining manufacturing data corresponding to the manufacturing machine; acquiring operator data corresponding to an operator operating the manufacturing machine; obtaining material data corresponding to materials being used by at least one of the operator or the manufacturing machine in the manufacturing process, the data including a material usage rate; calculating a calculation data set based on the manufacturing data, the operator data, and the material data; and a processor configured to analyze 4M, which are man, machine, material and method data, and perform analysis on at least the operator data based on the results of the computational data set to prevent material waste by a particular operator.

20. The system of claim 19 , wherein the processor is configured to provide as output a user interface including results of the analysis of at least the operator data based on the results of the computational data set.

Images