Method and system for visualizing four-dimensional space-time sensor data on basis of time-space matching of time series sensor data

Abstract

Provided are a method and system for visualizing four-dimensional space-time sensor data on the basis of time-space matching of time series sensor data. In a method for visualizing a 3D printing process state, according to an embodiment of the present invention, printing coordinate data and sensor data, which are collected in time series during a 3D printing process, are visualized by configuring same as space-time sensor data through time-space matching. Accordingly, it is possible to temporally and spatially identify the 3D printing process state without expensive equipment, and analyze a defect occurred time point, location, and cause, thereby contributing to improving 3D printing quality and reducing a defect rate.

Description

Method and System for Visualizing 4D Spatiotemporal Sensor Data Based on Time-Space Matching of Time-Series Sensor Data

[0001] The present invention relates to 3D printing process monitoring, and more specifically, to a method for processing and visualizing time-series sensor data generated during a 3D printing process so that it is useful for analyzing process anomalies.

[0002] In metal 3D printing processes, micropores, cracks, and non-fusion defects are prone to occur during the repetitive laser melting process, which can lead to a degradation of product performance. Accordingly, as shown on the left side of Figure 1, non-destructive inspection techniques such as X-ray CT and thermal imaging cameras have been primarily used to detect defects occurring during the process. However, these methods have limitations in that they require expensive equipment and require long analysis times, making real-time response impossible.

[0003] Accordingly, as shown on the right side of Figure 1, research has recently been attempted to predict defects during the process through sensor-based monitoring and machine learning techniques. However, since sensor data contains only time information, there are limitations in accurately identifying the location of defects or visualizing and interpreting abnormal signs by integrating them spatiotemporally through analysis limited to the time axis.

[0004] Accordingly, measures are needed to enable spatiotemporal data analysis without the need for expensive equipment.

[0005] The present invention has been devised to solve the above-mentioned problems, and the objective of the present invention is to provide a method for visualizing the state of a 3D printing process in space and time without expensive equipment, by configuring printing coordinate data and sensor data collected in a time series during the 3D printing process into space-time sensor data through time-space alignment.

[0006] In addition, another objective of the present invention is to provide a method for visualizing a 3D printing process state capable of detecting and analyzing the occurrence of anomalies in the 3D printing process by generating and visualizing correlation patterns between spatiotemporal sensor data.

[0007] A method for visualizing a 3D printing process state according to an embodiment of the present invention for achieving the above objective comprises: a step in which a 3D printing process state visualization system collects time-series printing coordinate data during a 3D printing process; a step in which a 3D printing process state visualization system collects time-series sensor data during a 3D printing process; a step in which a 3D printing process state visualization system uses the collected data to configure each sensor data into spatiotemporal sensor data including a time stamp, printing coordinate data, and a sensor value; a step in which a 3D printing process state visualization system stores the configured spatiotemporal sensor data; and a step in which a 3D printing process state visualization system visualizes the stored spatiotemporal sensor data.

[0008] The time-series printing coordinate data collection step can be collected only during the time interval in which 3D printing is performed.

[0009] Time series printing coordinate data includes a time stamp and printing coordinate data, and time series sensor data includes a time stamp and sensor value, and the configuration step can configure spatiotemporal sensor data by aligning the printing coordinate data of the time series printing coordinate data and the sensor value of the time series sensor data based on the time stamp.

[0010] The visualization step can visualize the sensor values ​​of the spatiotemporal sensor data on the corresponding printing coordinates.

[0011] A method for visualizing a 3D printing process state according to an embodiment of the present invention may further include the step of a 3D printing process state visualization system converting the sensor value of configured spatiotemporal sensor data into a color scale.

[0012] A method for visualizing a 3D printing process state according to an embodiment of the present invention may further include the step of a 3D printing process state visualization system interpolating time-series sensor data to match the time stamp of time-series printing coordinate data.

[0013] A method for visualizing a 3D printing process state according to an embodiment of the present invention may further include the step of a 3D printing process state visualization system extracting cross-sectional data of spatiotemporal sensor data and visualizing it.

[0014] A method for visualizing a 3D printing process state according to an embodiment of the present invention may further include: a step in which a 3D printing process state visualization system calculates correlation coefficients between stored spatiotemporal sensor data; a step in which a 3D printing process state visualization system sorts the calculated correlation coefficients to generate a correlation coefficient matrix; a step in which a 3D printing process state visualization system generates a correlation coefficient pattern by differently expressing correlation coefficients that exceed a threshold value in the correlation coefficient matrix; and a step in which a 3D printing process state visualization system visualizes the generated correlation coefficient pattern.

[0015] The correlation coefficient matrix generation step can generate a correlation coefficient matrix by selecting some sensor data based on correlation coefficients and aligning the correlation coefficients among the selected sensor data.

[0016] A 3D printing process state visualization system according to another embodiment of the present invention comprises: a communication unit that collects time-series printing coordinate data and time-series sensor data during a 3D printing process; a processor that uses the collected data to configure each sensor data into spatiotemporal sensor data including a time stamp, printing coordinate data, and sensor value; a storage unit that stores the configured spatiotemporal sensor data; and an output unit that visualizes the stored spatiotemporal sensor data.

[0017] A method for visualizing a 3D printing process state according to another embodiment of the present invention comprises: a step in which a 3D printing process state visualization system uses time-series printing coordinate data and time-series sensor data collected during a 3D printing process to configure each sensor data into spatiotemporal sensor data including a time stamp, printing coordinate data, and sensor value; a step in which the 3D printing process state visualization system calculates correlation coefficients between the configured spatiotemporal sensor data; a step in which the 3D printing process state visualization system sorts the calculated correlation coefficients to generate a correlation coefficient matrix; a step in which the 3D printing process state visualization system generates a correlation coefficient pattern by differently expressing correlation coefficients that exceed a threshold value in the correlation coefficient matrix; and a step in which the 3D printing process state visualization system visualizes the generated correlation coefficient pattern.

[0018] A 3D printing process state visualization system according to another embodiment of the present invention comprises: a processor that uses time-series printing coordinate data and time-series sensor data collected during a 3D printing process to configure each sensor data into spatiotemporal sensor data including a time stamp, printing coordinate data, and sensor value, calculates correlation coefficients between the configured spatiotemporal sensor data, sorts the calculated correlation coefficients to generate a correlation coefficient matrix, and generates a correlation coefficient pattern by differently expressing correlation coefficients that exceed a threshold value in the correlation coefficient matrix; and an output unit that visualizes the generated correlation coefficient pattern.

[0019] As explained above, according to the embodiments of the present invention, printing coordinate data and sensor data collected in a time series during a 3D printing process are visualized by configuring them into spatiotemporal sensor data through time-space alignment, thereby enabling the verification of the 3D printing process status in spatiotemporal terms without expensive equipment and the analysis of the time, location, and cause of defects, which can contribute to improving 3D printing quality and reducing the defect rate.

[0020] According to embodiments of the present invention, by generating and visualizing correlation patterns between spatiotemporal sensor data, it is possible to detect whether an abnormality has occurred in the 3D printing process and analyze the cause, and to establish a foundation for future integration with AI-based process abnormality detection and control.

[0021] FIG. 1 shows a conventional 3D printing defect and anomaly detection technique,

[0022] FIG. 2 is a method for visualizing the state of a 3D printing process according to an embodiment of the present invention,

[0023] Figure 3 shows a method for identifying the On section using the 2-central clustering method,

[0024] Figure 4 shows a method for identifying the On section by a threshold value setting method,

[0025] Figure 5 shows the alignment and visualization of time-series sensor data and printing coordinate data.

[0026] Figure 6 is an example of visualization of various spatiotemporal sensor data,

[0027] Figure 7 is an example of a cross-sectional data visualization result.

[0028] Figure 8 is an example of anomaly detection results in cross-sectional data,

[0029] FIG. 9 is a method for visualizing the state of a 3D printing process according to another embodiment of the present invention,

[0030] Figure 10 is an example of the selection results of spatiotemporal sensor data with high correlation coefficients.

[0031] Figure 11 is an example of the result of generating a correlation coefficient matrix,

[0032] FIGS. 12 to 14 are examples of correlation coefficient patterns,

[0033] Figures 15 to 17 are examples of correlation pattern visualizations,

[0034] FIG. 18 is a 3D printing process state visualization system according to another embodiment of the present invention.

[0035] The present invention will be described in more detail below with reference to the drawings.

[0036] An embodiment of the present invention presents a method and system for visualizing 4-dimensional spatiotemporal sensor data based on time-space alignment of time-series sensor data. This is a technology that visualizes printing coordinate data and sensor data collected in time series during a 3D printing process by configuring them into spatiotemporal sensor data through time-space alignment. Additionally, it generates and visualizes correlation patterns between spatiotemporal sensor data.

[0037] FIG. 2 is a diagram illustrating the flow of a method for visualizing the state of a 3D printing process according to an embodiment of the present invention.

[0038] In order to construct the 4-dimensional spatiotemporal sensor data required to visualize the 3D printing process state, as described above, time-series printing coordinate data is first collected during the 3D printing process (S110), and time-series sensor data are collected (S120).

[0039] The time-series printing coordinate data collected in step S110 is data representing the time-series change of the coordinate data of the point where the 3D printer prints, and consists of [time stamp (t), coordinate data (x,y,z)].

[0040] Time-series printing coordinate data can be collected by acquiring data representing the laser emission point from the nozzle of a 3D printer (for example, the driving value of a galvanometer mirror that controls the laser irradiation point).

[0041] Meanwhile, time-series printing coordinate data must be collected only during the On period, which is the time interval during which 3D printing is performed. To this end, photodiode (PD) sensor values ​​capable of detecting whether a laser for 3D printing is irradiated from the nozzle can be used, and there are two specific methods.

[0042] One is the K-means Clustering method, which divides PD sensor values ​​into two clusters as shown in Figure 3, and identifies the time intervals where relatively low values ​​appear as the Off intervals and the time intervals where relatively high values ​​appear as the On intervals.

[0043] Another method is a threshold setting method, as illustrated in FIG. 4, in which time intervals when PD sensor values ​​are below a threshold are identified as Off intervals, and time intervals when they are above a threshold are identified as On intervals. Here, the threshold can be set by dividing the PD sensor values ​​into two clusters and utilizing the mean and standard deviation for each.

[0044] Accordingly, in step S110, time series printing coordinate data can be collected only in the On section, excluding the Off section.

[0045] The time-series sensor data collected in step S120 consists of sensor data collected from various environmental sensors (e.g., temperature sensors, pressure sensors, gas concentration sensors, etc.) required for monitoring the 3D printing process. The time-series sensor data consists of [time stamp (t), sensor value (v)]. The time-series sensor data does not contain spatial information. That is, through the time-series sensor data, one can only know when the data was generated, and it is not possible to know which point the sensor data was generated at when printing.

[0046] In an embodiment of the present invention, time-series sensor data is matched with time-series printing coordinate data so that the time-series sensor data is linked with spatial information. Meanwhile, the time-series sensor data has a lower temporal resolution than the time-series printing coordinate data.

[0047] Accordingly, in order to minimize the alignment error in step S140 described below, time series sensor data are interpolated to match the time stamp (t) of the time series printing coordinate data prior to alignment (S130). In step S130, a linear interpolation technique may be used, but the use of other techniques is not excluded.

[0048] In the next step S130, the time series sensor data interpolated and the time series printing coordinate data collected in step S110 are aligned based on the time stamp to construct spatiotemporal sensor data [time stamp (t), coordinate data (x,y,z), sensor value (v)] (S140).

[0049] This allows spatial information to be added to time-series sensor data, making it possible to determine when (t) and at which point (x,y,z) the sensor value (v) was generated when printing, thereby enabling spatiotemporal analysis and visualization.

[0050] Meanwhile, when expressing the sensor value (v) in space, color is more appropriate than numerical value. Accordingly, the sensor value (v) in the spatiotemporal sensor data configured in step S140 is converted to the corresponding color scale (S150). By doing so, the sensor value (v) is converted from numerical value to color.

[0051] In the next step S150, spatiotemporal sensor data with sensor values ​​(v) converted into a color scale are stored in the data DB (S160). The spatiotemporal sensor data stored in step S160 can be visualized as spatiotemporal data in a color scale (S170).

[0052] Figure 5 schematically illustrates the process of visualizing spatiotemporal sensor data constructed by aligning time-series sensor data and time-series printing coordinate data through the process described so far. Meanwhile, visualization is performed individually for each spatiotemporal sensor data. That is, as shown in Figure 6, it is possible to visualize each of the various spatiotemporal sensor data (fume filter pressure, chamber pressure, chamber temperature, build plate temperature).

[0053] This enables the simultaneous identification of the time and spatial location where an anomaly first occurred during the 3D printing process, thereby allowing for precise tracking and analysis of the cause of the problem. Additionally, since the temporal flow and spatial changes of the entire 3D printing process can be visually reproduced, it can be effectively utilized for regression analysis of past process states and quality verification.

[0054] Meanwhile, since the spatiotemporal sensor data visualized by step S170 contains spatial information, it is also possible to extract and visualize cross-sectional data (S180). A cross-section is obtained by cutting the spatiotemporal sensor data perpendicularly along a specific axis in space (x-axis, y-axis, z-axis, or an axis created by combining these). Figure 7 illustrates the result of visualizing cross-sectional data. This enables not only an overall analysis of the spatiotemporal sensor data but also a local analysis of the cross-sectional area.

[0055] Furthermore, it is possible to detect outliers from cross-sectional data. Since the extracted cross-sectional data corresponds to a color image expressed in a color scale, it is possible to monitor the process status through image analysis. Specifically, the process status can be monitored by detecting signs of anomalies by identifying visual features such as abnormal color distribution, sudden color changes, local color anomalies, and continuous boundary distortion at specific locations. While this can be performed by a manager, automated detection of anomalies is also possible by applying image analysis algorithms or machine learning models. Figure 8 illustrates the results of anomaly detection.

[0056] FIG. 9 is a diagram illustrating the flow of a method for visualizing the state of a 3D printing process according to another embodiment of the present invention.

[0057] To visualize the 3D printing process status, first, spatiotemporal sensor data is generated during the 3D printing process and stored in a data DB (S210). Step S210 can be implemented through steps S110 to S160 of FIG. 2.

[0058] Subsequently, correlation coefficients between the spatiotemporal sensor data stored in step S210 are calculated (S220). The correlation coefficients are calculated based on the time data, utilizing the Pearson Correlation Coefficient to determine the correlation coefficients between each pair of spatiotemporal sensor data.

[0059] Through step S220, a combination of spatiotemporal sensor data that are closely related to each other can be generated in the 3D printing process. Specifically, a specific number of spatiotemporal sensor data are selected in order of highest correlation coefficient (S230). Figure 10 illustrates the result of selecting five data points with high correlation coefficients (oxygen concentration, chamber pressure, fume filter pressure, build plate temperature, chamber temperature) from a plurality of spatiotemporal sensor data.

[0060] In the next step S230, the correlation coefficients between the selected spatiotemporal sensor data are arranged into a matrix to generate a correlation coefficient matrix (S240). Figure 11 illustrates the result of generating the correlation coefficient matrix.

[0061] Subsequently, correlation coefficients that exceed a threshold set by the user in the correlation coefficient matrix are represented differently, such as by assigning a different color, to generate a correlation coefficient pattern (S250). The correlation coefficient pattern allows one to identify which of the major spatiotemporal sensor data showed high correlation coefficients.

[0062] Meanwhile, by cumulatively analyzing correlation coefficient patterns, it is possible to classify the correlation coefficient patterns when no 3D printing process abnormalities occur and when 3D printing process abnormalities occur. For instance, through experience and analysis, it can be determined that the correlation coefficient pattern in Fig. 12 mainly appears when no 3D printing process abnormalities occur, while the correlation coefficient patterns in Figs. 13 and 14 mainly appear when 3D printing process abnormalities occur. Furthermore, it can be determined that different correlation coefficient patterns appear depending on the cause of the process abnormality.

[0063] Therefore, by visualizing the correlation coefficient pattern generated in step S250, the manager can determine / analyze the 3D printing process status (S260). In this case, as shown in FIGS. 15 to 17, key spatiotemporal sensor data included in the correlation pattern can also be visualized together.

[0064] Furthermore, it is also possible to automatically determine the process status of 3D printing from the correlation coefficient pattern generated in step S250 using an automated analysis tool.

[0065] FIG. 18 is a diagram illustrating the configuration of a 3D printing process status visualization system according to another embodiment of the present invention. The 3D printing process status visualization system according to an embodiment of the present invention can be implemented as a computing system comprising a communication unit (310), an output unit (320), a processor (330), an input unit (340), and a storage unit (350) as illustrated.

[0066] The communication unit (310) is a communication interface for connecting to an external network or external device, and is connected to communicate with the 3D printer to collect printing coordinate data and sensor data in a time series during the 3D printing process.

[0067] The output unit (320) is an output means for displaying the result of an operation performed by the processor (330), and the input unit (340) is a user interface that receives user commands and transmits them to the processor (330).

[0068] The processor (330) configures spatiotemporal sensor data according to the procedure illustrated in FIGS. 2 and FIG. 9 described above, visualizes it through the output unit (320), and generates correlation patterns between spatiotemporal sensor data and visualizes them through the output unit (320). The storage unit (350) provides storage space necessary for the processor (330) to function and operate.

[0069] Up to now, a method and system for visualizing 4-dimensional spatiotemporal sensor data based on time-space alignment of time-series sensor data have been described in detail with preferred embodiments.

[0070] In the above embodiment, printing coordinate data and sensor data collected in a time series during the 3D printing process are visualized by configuring them into spatiotemporal sensor data through time-space alignment, thereby enabling the status of the 3D printing process to be checked spatiotemporally without expensive equipment and to analyze the time, location, and cause of defects, thus contributing to the improvement of 3D printing quality and the reduction of the defect rate.

[0071] In addition, by generating and visualizing correlation patterns between spatiotemporal sensor data, it is possible to detect whether anomalies occur in the 3D printing process and analyze their causes, and to establish a foundation for future integration with AI-based process anomaly detection and control.

[0072] Meanwhile, it goes without saying that the technical concept of the present invention may also be applied to a computer-readable recording medium containing a computer program that enables the device and method according to the present embodiment to perform their functions. Furthermore, the technical concept according to various embodiments of the present invention may be implemented in the form of computer-readable code recorded on a computer-readable recording medium. A computer-readable recording medium may be any data storage device that can be read by a computer and store data. For example, a computer-readable recording medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, hard disk drive, etc. Additionally, computer-readable code or a program stored on a computer-readable recording medium may be transmitted through a network connected between computers.

[0073] Furthermore, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. Various modifications are possible by those skilled in the art without departing from the essence of the invention as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present invention.

Claims

1. A step in which a 3D printing process status visualization system collects time-series printing coordinate data during the 3D printing process; A 3D printing process status visualization system, a step of collecting time-series sensor data during the 3D printing process; A 3D printing process status visualization system, using collected data, comprises the step of configuring each sensor data into spatiotemporal sensor data including a time stamp, printing coordinate data, and sensor values; A step in which a 3D printing process status visualization system stores configured spatiotemporal sensor data; A 3D printing process state visualization method characterized by including the step of a 3D printing process state visualization system visualizing stored spatiotemporal sensor data.

2. In Claim 1, The time-series printing coordinate data collection step is, A method for visualizing the state of a 3D printing process, characterized by collecting data only during the time interval in which 3D printing is performed.

3. In Claim 1, Time-series printing coordinate data, Includes timestamp and printing coordinate data, Time-series sensor data, Includes time stamps and sensor values, The configuration stage is, A method for visualizing the state of a 3D printing process characterized by aligning the printing coordinate data of time-series printing coordinate data with the sensor values ​​of time-series sensor data based on a time stamp to construct spatiotemporal sensor data.

4. In Claim 3, The visualization step is, A method for visualizing the state of a 3D printing process characterized by visualizing the sensor values ​​of spatiotemporal sensor data on the corresponding printing coordinates.

5. In Claim 4, A 3D printing process state visualization method characterized by further including the step of converting the sensor values ​​of configured spatiotemporal sensor data into a color scale.

6. In Claim 3, A 3D printing process state visualization method characterized by further including the step of interpolating time-series sensor data to match the time stamp of time-series printing coordinate data.

7. In Claim 1, A 3D printing process state visualization method characterized by further including the step of a 3D printing process state visualization system extracting and visualizing cross-sectional data of spatiotemporal sensor data.

8. In Claim 1, A 3D printing process status visualization system calculates correlation coefficients between stored spatiotemporal sensor data; A 3D printing process state visualization system, a step of generating a correlation coefficient matrix by aligning the calculated correlation coefficients; A 3D printing process state visualization system generates a correlation coefficient pattern by differently representing correlation coefficients that exceed a threshold in a correlation coefficient matrix; A 3D printing process state visualization method characterized by further including the step of visualizing a generated correlation coefficient pattern in a 3D printing process state visualization system.

9. In Claim 8, The correlation coefficient matrix generation step is, A 3D printing process state visualization method characterized by selecting some sensor data based on correlation coefficients and generating a correlation coefficient matrix by aligning the correlation coefficients between the selected sensor data.

10. A communication unit that collects time-series printing coordinate data during the 3D printing process and collects time-series sensor data during the 3D printing process; A processor that uses collected data to configure each sensor data into spatiotemporal sensor data including a time stamp, printing coordinate data, and sensor value; A storage unit for storing configured spatiotemporal sensor data; A 3D printing process status visualization system characterized by including an output unit that visualizes stored spatiotemporal sensor data.

11. A 3D printing process state visualization system, using time-series printing coordinate data and time-series sensor data collected during the 3D printing process, comprises a step of configuring each sensor data into spatiotemporal sensor data including a time stamp, printing coordinate data, and sensor value; A 3D printing process status visualization system calculates correlation coefficients between configured spatiotemporal sensor data; A 3D printing process state visualization system, a step of generating a correlation coefficient matrix by aligning the calculated correlation coefficients; A 3D printing process state visualization system generates a correlation coefficient pattern by differently representing correlation coefficients that exceed a threshold in a correlation coefficient matrix; A 3D printing process state visualization method characterized by including the step of visualizing a generated correlation coefficient pattern using a 3D printing process state visualization system.

12. A processor that uses time-series printing coordinate data and time-series sensor data collected during a 3D printing process to configure each sensor data into spatiotemporal sensor data including a time stamp, printing coordinate data, and sensor value, calculates correlation coefficients between the configured spatiotemporal sensor data, sorts the calculated correlation coefficients to generate a correlation coefficient matrix, and generates a correlation coefficient pattern by differently representing correlation coefficients that exceed a threshold value in the correlation coefficient matrix; and A 3D printing process state visualization system characterized by including an output unit that visualizes a generated correlation coefficient pattern.

Images