Data perception and process datamation modeling method for lean manufacturing of intelligent factory

Abstract

The invention discloses a data perception and process datamation modeling method for lean manufacturing of an intelligent factory, and the method comprises the following steps: configuring a data collection module, and collecting data in real time from a sensor of a production line, a machine vision system for detecting the surface defects of a product, a PLC controller, and an ERP system; the acquired data is transmitted to a data acquisition server through an industrial Ethernet, and data acquisition software running on the data acquisition server is responsible for analysis and storage; after the data is cleaned and sorted through a data fusion module, a mixed deep learning model of a convolutional neural network (CNN) and a long short-term memory (LSTM) network is adopted to process the sorted real-time data, and the mixed deep learning model outputs various triples for knowledge graph modeling, including entities, relationships and attribute information; constructing a knowledge graph based on the output of the deep learning model, and determining the relationship between entities; an intelligent query module is used for querying the knowledge graph in real time and interacting with a user in a natural language form; and a fault early warning module is adopted to monitor data in real time.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent manufacturing, and particularly relates to a data perception and process data modeling method for lean manufacturing in intelligent factories. Background Art

[0002] As a core component of Industry 4.0, the goal of intelligent factories is to achieve a high degree of automation and intelligence in the production process by integrating advanced information technology, intelligent equipment, and innovative management concepts.

[0003] Although existing intelligent manufacturing systems have achieved a certain degree of automation and intelligence in the production process, there are still some significant problems that limit the improvement of production efficiency and product quality. The main problems include data islands, difficult system integration, and lack of top-level design. Data islands refer to the inability to effectively share and integrate data between different departments or systems, forming isolated data storage and processing units, resulting in the inability to comprehensively utilize data, poor information transmission, and affecting the coordination and optimization of the production process. For example, there is a lack of effective association between order information in the production management system and equipment sensor data, making it impossible to conduct comprehensive production analysis and optimization. Difficult system integration means that it is difficult to achieve seamless docking and data exchange between different systems, exacerbating the phenomenon of information islands. Difficult system integration makes enterprises face high technical and economic costs when implementing intelligent manufacturing. There is a lack of standard interfaces between equipment and software systems from different suppliers, increasing the difficulty of integration and maintenance costs. Lack of top-level design means that enterprises do not have an overall plan and design when implementing intelligent manufacturing, resulting in poor coordination and consistency between various subsystems. Lack of top-level design makes enterprises prone to problems such as blind investment and duplicate construction during the promotion of intelligent manufacturing, and it is difficult to form a unified intelligent manufacturing system. Different departments act independently, lacking a global perspective, resulting in waste of resources and low efficiency.

[0004] The existence of these problems makes existing intelligent manufacturing systems have obvious deficiencies in aspects such as data standardization, heterogeneous data processing, production and sales coordination, system autonomy, and intelligence improvement, resulting in low efficiency in informatization and intelligent construction and low return on investment. Summary of the Invention

[0005] In view of the above problems, the technical problem to be solved by the present invention is to provide a data perception and process data modeling method for lean manufacturing in intelligent factories.

[0006] To solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A data perception and process data modeling method for lean manufacturing in intelligent factories, comprising the following steps:

[0008] Configure a data acquisition module to collect data in real time from sensors on the production line, a machine vision system for detecting surface defects of products, a PLC controller, and an ERP system. The collected data is transmitted to a data acquisition server via industrial Ethernet, and the data acquisition software running on the data acquisition server is responsible for parsing and storing it;

[0009] In the data fusion module, clean, standardize, enhance, align, and fuse the collected data;

[0010] Adopt a hybrid deep learning model of convolutional neural network (CNN) and long short-term memory network (LSTM) to process the organized real-time data. Among them, CNN is used to process sensor data and machine vision data to extract spatial features; LSTM is used to process PLC data, ERP data, and sensor time series data to capture time dependencies and dynamic changes. The hybrid deep learning model outputs various triples for knowledge graph modeling, including entity, relationship, and attribute information;

[0011] Based on the output of the deep learning model, construct a knowledge graph to clarify the relationships between entities;

[0012] Adopt an intelligent query module to query the knowledge graph in real time and interact with users in natural language form;

[0013] Adopt a fault warning module to monitor data in real time, identify anomalies and generate warning information; the warning information includes the specific values of the abnormal data, timestamps, and production line information that may be affected. The warning information is updated in real time through the knowledge graph and sent to relevant personnel automatically detected by the intelligent agent.

[0014] In a possible implementation, it further includes: integrating all monitoring data into a unified platform to achieve closed-loop management of data flow and control flow.

[0015] In a possible implementation, in the process of cleaning, standardizing, enhancing, aligning, and fusing the collected data, the cleaning includes:

[0016] Use the median filtering algorithm to process the noise in sensor data; for the order information in the ERP system, perform a data integrity check to ensure that all necessary fields including order numbers, batch numbers, and quantities are correctly filled;

[0017] For missing or abnormal data, perform a reverse lookup in the ERP system according to the timestamp and batch number to supplement the missing order numbers. If negative or unreasonable values appear in the quantity field, correct them according to the average or median of historical order data;

[0018] The data collected by the PLC controller includes the device operation status, control signals, and operation logs. Through an interpolation method based on a time window, the missing values in the time series of the device operation status data are filled to ensure the continuity and integrity of the time series data. For the device operation logs recorded by the PLC, key event extraction methods are used to identify changes in device startup, shutdown, faults, and operation modes, and redundant and irrelevant log information is removed.

[0019] In one possible implementation, the standardization in the cleaning, standardization, enhancement, alignment, and fusion of the collected data includes:

[0020] Sensor data standardization: Uniformly convert parameters including temperature, pressure, and humidity to standard units;

[0021] ERP system data standardization: Perform field renaming and unit conversion on data including orders, inventory, and process parameters;

[0022] Machine vision data standardization: Normalize the extracted image defect feature values to the same dimension range;

[0023] PLC data standardization: Normalize the device operation status and control signal data, and standardize the timestamps at the same time.

[0024] In one possible implementation, constructing a knowledge graph based on the output of the deep learning model includes: entity information, relationship information, multi-modal attribute data, event sequences, spatial location information, and inference assistance data.

[0025] In one possible implementation, the real-time query of the knowledge graph using the intelligent query module and the interaction with the user in natural language form include:

[0026] The natural language sentence input by the user is represented as a word sequence S = {w1, w1, w n} after word segmentation and vectorization. The natural language understanding NLU module uses the fine-tuned LLM model to perform semantic parsing on the input and convert it into a query intent I and slot information F, that is:

[0027] I, F = f NLU (S)

[0028] where f NLU represents the NLU function. The parsed intent I and slot F are input into the query generation module to generate a query statement Q for the knowledge graph:

[0029] Q = f QueryGen (I, F)

[0030] f QueryGenIt is a query generation function used to map the parsing results into an executable query language;

[0031] The generated query Q is executed on the knowledge graph KG to obtain the query result R:

[0032] R = KG(Q)

[0033] The knowledge graph KG consists of a series of triples (s i , p i , o i ), where s i is the head entity, p i is the relationship, o i is the tail entity. The result processing and summarization module processes R to obtain the refined result R':

[0034] R' = f Process (R)

[0035] The f Process function is used for operations including data cleaning, information summarization, and result sorting;

[0036] The processed result R' is input into the natural language generation NLG module to generate a user-readable natural language response S resp :

[0037] S resp = f NLG (R')

[0038] The f NLG function uses a fine-tuned LLM model to convert structured data into fluent natural language.

[0039] Adopting the present invention has the following beneficial effects:

[0040] (1) Solve the data silo problem and enhance the data sharing and integration capabilities. By integrating multi-source data from sensors, device logs, ERP, and MES systems, data collection, cleaning, fusion, and standardization processing are achieved. On this basis, the problem of scattered data between departments and ineffective sharing in traditional intelligent factories is solved, enabling seamless integration and data flow between different systems. This data integration method provides a reliable data foundation for subsequent production optimization and decision support, while reducing information transmission barriers and improving the comprehensive utilization rate of factory data resources.

[0041] (2) Enhance the transparency and visualization of production management. The introduction of the knowledge graph has structured and correlated information such as the production equipment, product orders, and process flows in the factory, and presented it in a graphical way, achieving global transparency in production management. Operators can quickly query the production history, order status, or fault records of a certain piece of equipment through the knowledge graph, while managers can easily grasp the overall operation status of the factory. The intuitive visualization management method not only improves the efficiency of information acquisition but also reduces the difficulty of understanding complex data through manual intervention.

[0042] (3) Improve the ability of anomaly detection and predictive maintenance. With the help of the real-time analysis of the knowledge graph in the production process by intelligent agents, it is possible to accurately identify abnormal patterns in equipment operation and give early warnings of possible faults. For example, by analyzing parameters such as the vibration frequency and temperature of a rolling mill, it is possible to predict the possible damage time of its bearings, thus suggesting a maintenance window. In this way, the factory effectively avoids downtime or product unqualified situations caused by sudden equipment failures, significantly reducing the maintenance cost and the risk of production interruption.

[0043] (4) Optimize the production process and improve resource utilization. By using the analysis ability of deep learning models combined with production data, it is possible to identify bottlenecks or inefficient links in the process and give optimization suggestions. For example, in stainless steel production, the model can suggest adjusting the smelting time or temperature parameters, thus reducing energy consumption and raw material waste. At the same time, through accurate analysis of equipment operation data, the factory can dynamically optimize the production line scheduling to maximize equipment utilization. This data-driven process optimization method effectively reduces the production cost per unit product and improves the overall production efficiency.

[0044] (5) Modular design, supporting flexible deployment and expansion. Adopting a modular design, covering functional modules such as data collection, fusion, analysis, knowledge graph construction, and intelligent query, users can flexibly deploy relevant functional modules according to their own needs. For example, a small factory can only deploy data collection and fusion modules, while a large factory can expand the knowledge graph and deep learning analysis functions. Brief Description of the Drawings

[0045] Figure 1 It is a step flow chart of a data perception and process data modeling method for lean manufacturing in an intelligent factory according to an embodiment of the present invention. Detailed Embodiment

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] See Figure 1 , which shows a step flowchart of a data perception and process data modeling method for lean manufacturing in an intelligent factory according to an embodiment of the present invention, including the following steps:

[0048] S10. Configure a data acquisition module to collect data in real time from sensors on the production line (such as temperature and pressure sensors), a machine vision system for detecting surface defects of products, a PLC controller, and an ERP system (order and inventory information). The collected data is transmitted to a data acquisition server through industrial Ethernet, and the data acquisition software running on the data acquisition server is responsible for parsing and storing it;

[0049] S20. In the data fusion module, clean, standardize, enhance, align, and fuse the collected data;

[0050] S30. Use a hybrid deep learning model of a convolutional neural network (CNN) and a long short-term memory network (LSTM) to process the real-time data after arrangement. Among them, CNN is used to process sensor data and machine vision data to extract spatial features; LSTM is used to process PLC data, ERP data, and sensor time series data to capture time dependencies and dynamic changes. The hybrid deep learning model outputs various triples for knowledge graph modeling, including entity, relationship, and attribute information;

[0051] S40. Based on the output of the deep learning model, construct a knowledge graph to clarify the relationships between entities;

[0052] S50. Use an intelligent query module to query the knowledge graph in real time and interact with the user in natural language form;

[0053] S60. Use a fault warning module to monitor data in real time, identify anomalies, and generate warning information; the warning information includes the specific values, timestamps of the abnormal data, and the production line information that may be affected. The warning information is updated in real time through the knowledge graph and is automatically detected by an intelligent agent and sent to relevant personnel.

[0054] In a specific application example, the cleaning in cleaning, standardizing, enhancing, aligning, and fusing the collected data includes:

[0055] The median filtering algorithm is used to process the noise in sensor data. For example, for temperature sensor data, 10 consecutive temperature values are read, and the median of these values is calculated as the temperature value at the current time point to reduce the impact of random noise. For the order information (such as order number, batch number, quantity, etc.) in the ERP system, data integrity checks are performed to ensure that all necessary fields including the order number, batch number, and quantity are correctly filled;

[0056] For missing or abnormal data, reverse lookup is performed in the ERP system based on the timestamp and batch number to supplement the missing order number. If negative or unreasonable values appear in the quantity field, they are corrected according to the average or median of historical order data;

[0057] For the image data collected by the machine vision system, the main tasks are to remove noise, enhance image quality, and extract target features. Through image denoising algorithms such as Gaussian filtering and mean filtering, the random noise in the image can be effectively reduced, improving the data quality. In addition, in order to more accurately identify product surface defects, image enhancement techniques such as histogram equalization are used to adjust the brightness and contrast of the image to ensure that details are clearly visible. During the cleaning process, duplicate or incomplete image data also need to be removed to ensure the integrity and accuracy of the data input into the deep learning model. After image processing, edge detection algorithms and convolutional neural networks (CNNs) are used for feature extraction, converting the defect information (such as scratches, cracks, oxidation spots, etc.) in the image into structured data for subsequent analysis and prediction.

[0058] The data collected by the PLC controller includes device operating status, control signals, and operation logs. These data usually appear in the form of time series. During the data cleaning process, it is necessary to address possible outliers, missing values, and duplicate records in the data. Through an interpolation method based on a time window, the missing values in the time series of device operating status data are filled to ensure the continuity and integrity of the time series data; for the device operation logs recorded by the PLC, key event extraction methods are used to identify the start and stop of the device, faults, and changes in the operating mode, and redundant and irrelevant log information is removed. When processing control signal data, the signal amplitudes of different devices are adjusted to the same dimension range through a normalization method for fusion analysis with other data. The PLC data after cleaning and processing can not only reflect the real-time operating status of the device but also provide time series inputs closely related to device faults and production efficiency for the deep learning model, thus improving the accuracy and prediction ability of overall data analysis.

[0059] After data collection, the data is standardized to ensure the consistency of data from different sources in terms of format and unit, laying a foundation for subsequent data fusion, analysis, and model training. In a specific application example, during the processes of cleaning, standardizing, enhancing, aligning, and fusing the collected data, the standardization includes:

[0060] Sensor data standardization: Uniformly convert parameters including temperature, pressure, and humidity into standard units to ensure data consistency, and standardize the timestamp format to the standard time format of "YYYY-MM-DD HH:MM:SS";

[0061] ERP system data standardization: Rename fields and convert units for data including orders, inventory, and process parameters to ensure the uniformity of data from different sources in terms of format and unit, facilitating fusion processing with other data;

[0062] Machine vision data standardization: Normalize the numerical values of extracted image defect features (such as area and shape) to the same dimension range to ensure that the feature data can be directly used for subsequent model analysis, and unify the time identification and defect type coding;

[0063] PLC data standardization: Normalize the device operation status and control signal data to ensure the consistency of dimensions for different devices and parameters. At the same time, standardize the timestamp to ensure the unity and comparability of time series data.

[0064] In practical applications, the problem of dataset imbalance often affects the training effect of the model. Especially when the number of samples of a certain type of defective product is much less than that of normal products during the production process, the prediction performance of the model may be greatly affected. To solve this problem, using the Synthetic Minority Over-sampling Technique (SMOTE) to process the imbalanced dataset during the data enhancement process after collection is an effective solution. The SMOTE algorithm balances the dataset by generating new minority class samples, thereby improving the model's learning ability for the minority class and avoiding the overfitting problem brought by traditional oversampling methods (such as replicating samples). The specific operation steps are as follows:

[0065] Select minority class samples: Select samples belonging to the minority class from the dataset. Usually, by calculating the distribution frequencies of various samples, identify the classes with fewer samples. A threshold can be set to determine which classes belong to the "minority class".

[0066] Find the nearest neighbor samples: For each minority-class sample, calculate its similarity to other samples. Commonly used distance metrics include Euclidean distance or Manhattan distance. By calculating the distances between samples, the K nearest neighbor samples can be found. To improve the diversity of the generated samples, different distance metric methods can be selected to ensure that the distance calculation accurately reflects the similarity between data.

[0067] Generate synthetic samples: Randomly select a sample N from the selected nearest neighbor samples, and interpolate between the current minority-class sample and the neighbor sample to generate a new sample. The formula is:

[0068] S = X_sample + λ · (X_neighbor - X_sample)

[0069] where λ is a random number uniformly distributed between [0, 1], and X_sample and X_neighbor represent the feature vectors of the current sample and the selected neighbor sample respectively. The generated new sample S is located between these two samples and has the characteristics of the minority class.

[0070] Repeat the generation process: Repeat the above steps until a sufficient number of synthetic samples are generated so that the number of minority-class samples is comparable to that of majority-class samples. Usually, the amount of synthetic samples generated can be set according to specific requirements, such as increasing the number of minority-class samples to be equal to that of majority-class samples, or setting a certain increase ratio.

[0071] In the data fusion process, the SMOTE algorithm is particularly important when dealing with sensor data and PLC data. In sensor data, the number of abnormal state samples (such as abnormal temperature or pressure) is usually much less than that of normal state samples. By generating new abnormal samples through SMOTE, the data distribution is balanced, and the model's ability to recognize abnormal patterns is improved. For PLC data, especially in the minority states of equipment operation failures, SMOTE can expand the abnormal time segment samples and effectively enhance the generalization performance of the fault prediction model.

[0072] In the post-acquisition data augmentation process, image feature enhancement technology is a method of adjusting the visual characteristics of images to highlight target features, and is commonly used in image data preprocessing and model input optimization. Its core methods include histogram equalization, edge detection, and multi-scale feature extraction, etc. Among them, histogram equalization can adjust the pixel brightness distribution to improve the contrast and detail visibility of the image, and its mathematical expression is:

[0073]

[0074] where, p k is the probability distribution of pixel value k, c k is the corresponding equalized pixel value, L is the gray level, nk m and N are the number of pixels with gray value k and the total number of pixels, respectively.

[0075] When processing machine vision data, this technology is used to optimize the visualization effect of defect images. For example, the detected surface defect image is first subjected to histogram equalization to improve the contrast of the defect area, and then edge detection technology (such as the Canny algorithm) is applied to locate the defect boundary, and features including the area and position of the defect are extracted. These features are structured into formalized data (such as feature matrix F = {f1, f2, …, f n}), and are associated with the production batch number and process parameters to provide high-quality input data for subsequent model analysis.

[0076] In the data fusion process, when processing machine vision data, the image feature enhancement technology first optimizes the contrast of the product surface defect image, eliminates the influence of noise, and then uses edge detection technology to accurately extract the geometric features (such as area, shape, and position) of the defect area, converts them into structured data, and finally associates them with the order number and batch number of the ERP system. This processing method effectively improves the usability of visual data and provides high-quality input for subsequent fusion analysis.

[0077] In the data enhancement process after acquisition, the time series interpolation and alignment technology is a method for processing multi-source asynchronous time series data, which can achieve the integrity and consistency of data through interpolation and time synchronization. The interpolation method usually uses linear interpolation, and its formula is:

[0078]

[0079] where, v i represents the interpolated value, t j , t k are adjacent timestamps, and v j , v k are the corresponding original data values.

[0080] In the data fusion process, the time series interpolation and alignment technology plays a key role in processing PLC data and sensor data. There are often problems of inconsistent acquisition frequencies between the PLC device control data and the real-time monitoring data such as the temperature and pressure of the sensor. The two data are aligned through the time interpolation method, and the missing values are filled to form a complete and synchronized time series data set. In addition, the order time and batch number information of the ERP system are also associated with the device data through time alignment to ensure the consistency of multi-source data in the time dimension.

[0081] Deep learning models are trained to output after fusing data, and the output form is triples required for constructing a knowledge graph. For example, device status and fault information are extracted from sensor data and PLC data to form a triple <Device A, Status, Abnormal>. These triples structure the information of devices, products, faults, and production processes, providing a basis for the subsequent construction and query of the knowledge graph, and supporting intelligent fault diagnosis, equipment maintenance, and production process optimization. The input data for the deep learning model are: sensor data on the production line, including environmental parameters such as temperature, pressure, and humidity, as well as the operating status data of equipment; production process data recorded by PLC (Programmable Logic Controller), including production batches, product specifications, and production times; order information, inventory data, and process parameters in the ERP system; product surface image data collected by the machine vision system for detecting defect features. Historical fault records and maintenance data are also incorporated into the input for training the model to identify fault patterns. The deep learning model adopts a hybrid model of convolutional neural network (CNN) and long short-term memory network (LSTM). CNN is used to process sensor data and machine vision data to extract spatial features; LSTM is used to process PLC data, ERP data, and sensor time series data to capture time dependencies and dynamic changes. Finally, the model outputs various triples for knowledge graph modeling, including entity, relationship, and attribute information, providing data support for subsequent intelligent queries and fault warnings.

[0082] The formula for the CNN convolutional layer is:

[0083]

[0084] Among them, represents the output of the k-th convolutional kernel at the (i, j) position, f is the activation function, is the weight of the convolutional kernel, is the value of the c-th channel of the input data at the (i + m, j + n) position, b k is the bias term, M and N are the height and width of the convolutional kernel respectively, C in is the number of channels of the input data.

[0085] The formula for the LSTM cell is:

[0086] Input gate: i t = σ(W xi x t + W hi h t-1 + W ci ⊙ c t-1 + b i )

[0087] Forget gate: f t = σ(W xf xt +W hf h t-1 +W cf ⊙c t-1 +b f )

[0088] Output gate: o t = σ(W xo x t +W ho h t-1 +W co ⊙c t +b o )

[0089] Cell state update: c t = f t ⊙c t-1 + i t ⊙ tanh

[0090] Hidden state update: h t = o t ⊙ tanh

[0091] where σ is the sigmoid activation function, ⊙ represents element-wise multiplication, W and b are the weight and bias terms respectively, x t is the input at the current time step, h t-1 is the hidden state at the previous time step, c t-1 and c t are the cell states at the previous time step and the current time step respectively.

[0092] During the model training process, historical data is used as the training set, and the model parameters are optimized through the backpropagation algorithm (including optimization methods such as gradient descent) to enable it to accurately identify abnormal patterns.

[0093] After being processed by the combined CNN and LSTM hybrid model, the data output by the deep learning model is in the form of a structured representation of multimodal information (such as entities, relationships, time series prediction results), and these results are further passed to the knowledge graph modeling module for generating highly relevant structured data, mainly including entity information, relationship information, multimodal attribute data, event sequences, spatial location information

[0094] Entity information is the core entity extracted from the input data and is used to construct the nodes of the knowledge graph, including:

[0095] Device entities: such as sensor numbers, device IDs, production line numbers.

[0096] Environmental entities: such as the specific values of environmental parameters such as temperature, pressure, and humidity.

[0097] Product entities: such as product specifications, production batches, process parameters, etc. Fault entities: such as fault types (e.g., overheating, overload, etc.), time and location of fault occurrence.

[0098] In a knowledge graph, the relationships between entities form the key to knowledge modeling. The relationship information of the output data should include:

[0099] Temporal relationship: such as "production time -> product batch", indicating that a certain product is completed in a certain time period.

[0100] Dependency relationship: such as "equipment A -> sensor B", indicating that sensor B belongs to equipment A.

[0101] Causal relationship: such as "temperature anomaly -> fault C", indicating that a certain fault may be caused by temperature anomaly.

[0102] Multi-modal attribute data is used to attach its multi-modal attributes to each entity to form a complete node attribute table. Specifically, it includes:

[0103] Numerical attributes: such as continuous variables like temperature, pressure, humidity, etc.

[0104] Categorical attributes: such as equipment operating status (normal / abnormal), product quality (qualified / unqualified).

[0105] Textual attributes: such as maintenance logs, fault descriptions.

[0106] Timestamp attributes: such as production start time, anomaly detection time.

[0107] Event sequences are used to model event sequence information of dynamic temporal knowledge. The output includes:

[0108] Event types: such as "sensor data acquisition", "fault occurrence", "equipment maintenance", etc.

[0109] Event time: such as the start and end time of each event.

[0110] Event association: such as the equipment or production batch affected by a certain event.

[0111] Spatial location information: For the spatial correlation that needs to be modeled, the location data of equipment or sensors is output. It includes:

[0112] Physical location coordinates of equipment (such as specific areas within a factory).

[0113] Production line layout relationship (such as connection of upstream and downstream equipment).

[0114] Inference auxiliary data is auxiliary information used to support rule inference in a knowledge graph, including:

[0115] Rule data: such as "pressure > X and temperature > Y -> may cause fault Z".

[0116] Probability information: such as "the probability of abnormal device operation is 85%".

[0117] Convert the above data into a standard format suitable for knowledge graph modeling, including:

[0118] (1) Triple format: such as <entity A, relationship R, entity B>, for example: <production batch 001, produced on, 2024-11-21 10:00:00>; <device A, temperature anomaly, 85°C>; <product P1, uses device, device A>.

[0119] (2) Attribute description: such as <entity, attribute, value>, for example: <sensor 1, temperature, 85°C>; <device A, status, abnormal>.

[0120] A specific application example. In S40, the knowledge graph construction module uses the output of the deep learning model to create entities (such as devices, products, orders) and relationships (such as "produced", "needs maintenance"). For example, if the model predicts that a certain machine is about to fail, the knowledge graph will be updated to show the relationship between this machine and a specific order and marked as "needs maintenance".

[0121] Among them, the entity definitions include:

[0122] Device entity: includes various machines and devices on the production line, such as injection molding machines, cutting machines, assembly lines, etc.

[0123] Product entity: represents various products on the production line, including product batches, specifications, production dates, etc.

[0124] Order entity: represents customer order information, including order numbers, product quantities, delivery dates, etc.

[0125] The relationship definitions include:

[0126] "Produced": indicates which product the device produced.

[0127] "Needs maintenance": indicates that the device needs maintenance, as well as the maintenance priority and estimated time.

[0128] "Causes failure": indicates that a certain abnormality or event causes the device to fail.

[0129] The knowledge graph construction process includes: extracting entity and relationship information from the output of the deep learning model; using a graph database (such as Neo4j) to store entities and relationships to construct the knowledge graph; dynamically updating the relationship information in the knowledge graph according to the output results of the model, such as the maintenance status of the device, the production status of the product, etc.

[0130] Suppose in a specific application example of a stainless steel pipe manufacturing factory, the output of the deep learning model shows that there is a high probability of a fault occurring in a rolling mill A within the next 24 hours. Then the knowledge graph construction module will perform the following operations:

[0131] Extract entities: Identify the entities "rolling mill A" and "order X" (assuming the steel pipes being processed by rolling mill A belong to order X);

[0132] Extract relationships: Determine the relationship between "rolling mill A" and "order X" as "processed";

[0133] Update relationship information: In the knowledge graph, mark the relationship between "rolling mill A" and "needs maintenance" as high priority, and set the estimated maintenance time to a certain time point before the predicted fault time;

[0134] Dynamic update: If rolling mill A actually fails and is maintained, the knowledge graph will update the status of "rolling mill A" to "under maintenance", and record the maintenance completion time;

[0135] In this way, the knowledge graph can reflect the dynamic relationships among equipment, steel pipe products, and orders in real time, providing accurate support for production management and equipment maintenance decisions.

[0136] Furthermore, in a specific application example, the intelligent query module is an intelligent agent (agent) with a fine-tuned open-source large language model (LLM) as the core, customized specifically for the factory dataset, aiming to achieve real-time queries of the knowledge graph and interact with users in natural language form. Through a rigorous process design, this module ensures the efficiency and accuracy of the entire process from user input to information retrieval and then to natural language response. The real-time query of the knowledge graph by the intelligent query module and the interaction with users in natural language form include:

[0137] The natural language sentence input by the user is represented as a word sequence S = {w1, w1, w n} after word segmentation and vectorization. The natural language understanding NLU module uses the fine-tuned LLM model to perform semantic parsing on the input and convert it into a query intent I and slot information F, that is:

[0138] I,F = f NLU (S)

[0139] where f NLU represents the NLU function, and the parsed intent I and slot F are input into the query generation module to generate a query statement Q for the knowledge graph:

[0140] Q = f QueryGen (I,F)

[0141] f QueryGen is a query generation function used to map the parsing result into an executable query language (such as SPARQL);

[0142] The generated query Q is executed on the knowledge graph KG to obtain the query result R:

[0143] R = KG(Q)

[0144] The knowledge graph KG consists of a series of triples (s i , p i , o i ), where s i is the head entity, p i is the relationship, o i is the tail entity. Since the directly obtained result may contain redundant or complex information, the result processing and summarization module processes R to obtain the refined result R':

[0145] R' = f Process (R)

[0146] f Process The function is used for operations including data cleaning, information summarization, and result sorting;

[0147] The processed result R' is input into the natural language generation NLG module to generate a user-readable natural language response S resp :

[0148] S resp = f NLG (R')

[0149] f NLG Utilize the fine-tuned LLM model to convert structured data into fluent natural language.

[0150] The entire process can be summarized by the following formula:

[0151] S resp = f NLG (f Process (KG(f QueryGen (f NLU (S)))))

[0152] This formula describes the working process of the intelligent query module, covering key steps such as semantic parsing, query generation, knowledge retrieval, result processing, and language generation, from user input to natural language response.

[0153] An intelligent query module is introduced, which supports real-time query of the constructed knowledge graph and presents the query results to users in a concise and intuitive form through natural language processing (NLP) technology. Users input their query intentions in natural language, and the system parses and generates corresponding query statements, quickly providing key information such as device status, production progress, and fault warnings. The intelligent query module uses an open-source LLM model fine-tuned based on factory datasets, enabling the model to deeply understand the professional terms and specific contexts in the factory domain, improving the accuracy of semantic parsing and language generation. At the same time, the fine-tuned model shows higher reliability and security when processing internal factory data. In practical applications, the intelligent query module can help users quickly obtain the required information. For example, when a user asks about the real-time status of a certain production line, the module can understand the query intention, generate appropriate query statements, retrieve relevant data in the knowledge graph, and reply to the user in natural language after processing. This process is not only efficient but also ensures the accuracy and timeliness of information. The intelligent query module realizes the organic combination of natural language processing and knowledge graph technology, providing technical support for the intelligent and digital transformation of factories. The design of the module takes into account the user experience, ensuring a natural and smooth interaction process while guaranteeing the rigor of data processing and the reliability of results.

[0154] In a specific application example, for instance, an operator can ask: "Which orders were delayed due to the failure of equipment No. 3?" The intelligent query module will parse this natural language, convert it into graph database query information, start calling the retrieval tool, retrieve relevant information from the knowledge graph, and present the query results to the operator in a fluent natural language form for easy understanding by the operator. For example, the reply could be: "Order #123 was delayed due to the failure of equipment No. 3 on May 12, 2023."

[0155] A data perception and process data modeling method for lean manufacturing in an intelligent factory according to another embodiment of the present invention. Continuing to refer to Figure 1 , on the basis of S10 to S60, it further includes: S70, integrating all monitoring data into a unified platform to achieve closed-loop management of data flow and control flow. This platform provides a user interface that displays real-time data, historical trends, warning information, and query results. Operators can monitor the entire production process through this platform and make operation adjustments according to the suggestions provided by the intelligent agent. For example, if the intelligent agent suggests adjusting the machine settings to improve efficiency, the operator can directly execute these adjustments on the platform.

[0156] Next, taking a stainless steel factory as a specific application scenario, the implementation process of the method of the embodiment of the present invention will be described. A stainless steel factory is a typical scenario that combines discrete and process manufacturing. Its production process involves multiple processes, such as smelting, rolling, heat treatment, and finishing. The production equipment is complex, and the data sources are diverse, including sensor data, production plan data, and quality inspection data, etc. To achieve the goal of lean manufacturing, the following are the specific steps to implement this solution taking the stainless steel factory as an example.

[0157] Step 1: Data collection and preprocessing

[0158] 1.1 Configuration of data collection module

[0159] To comprehensively grasp the key data in the production process, first deploy and configure the data collection equipment in the stainless steel factory:

[0160] Sensor network: Install sensors in smelting furnaces, rolling mills, cooling equipment, and quality inspection stations to monitor parameters such as temperature, pressure, vibration, and surface defects in real time.

[0161] Vision inspection system: Deploy machine vision equipment in the quality inspection link to detect surface defects of products, such as scratches and oxidation spots.

[0162] Equipment control system: Collect equipment operation data from PLC (programmable logic controller), including start frequency, operation time, and shutdown status.

[0163] Production management system: Obtain order information, material inventory, process parameters, and production plans from ERP and MES systems.

[0164] The collected data is transmitted to the central data collection server through industrial Ethernet, and data access is realized through a unified interface specification.

[0165] 1.2 Data cleaning and standardization

[0166] After data collection, it is necessary to clean and standardize the data to improve data quality. Perform sensor data cleaning, ERP system data cleaning, machine vision data processing, and PLC data processing.

[0167] Step 2: Data fusion and balancing

[0168] 2.1 Data matching and association

[0169] Associate different data sources through batch numbers, equipment numbers, etc. For example: Associate the sensor data of the rolling mill with the production batch in the ERP system. And combine the defect records detected by machine vision with the process parameters of the corresponding production line.

[0170] 2.2 Dataset balancing

[0171] During the stainless steel production process, the number of defective samples may be significantly less than that of normal samples. The SMOTE algorithm is used to generate minority class samples to balance the data distribution. Specific operation: Select a record from the defective samples and find its K nearest neighbor samples. Generate random data points between the samples until enough samples are generated to achieve balance.

[0172] 2.3 Data storage

[0173] Store the cleaned, standardized, and fused data in a central database and provide data support for subsequent analysis modules through a standardized interface.

[0174] Step 3: Deep learning model analysis

[0175] 3.1 Model training

[0176] For the process characteristics of stainless steel factories, a deep learning model combining convolutional neural network (CNN) and long short-term memory network (LSTM) is adopted:

[0177] Input data: Include equipment sensor parameters (such as temperature, pressure, vibration frequency), historical defect data, and production plans.

[0178] Model function: CNN is used to extract the spatial features of sensor data (such as abnormal temperature distribution). LSTM is used to capture the long-term dependence relationships of time series data (such as the correlation between temperature changes and failure times).

[0179] Training set: Train the model based on historical data and use the cross-validation method to evaluate the model performance.

[0180] 3.2 Real-time data analysis

[0181] Input the real-time data into the trained model for data fusion.

[0182] Step 4: Knowledge graph construction and application

[0183] 4.1 Knowledge graph construction

[0184] According to the model output results, construct a knowledge graph containing entities such as equipment, products, orders, and their relationships. For example:

[0185] Entities: Equipment - Rolling mill A, Smelting furnace B. Product - Stainless steel plate #12345. Order: Order #56789.

[0186] Relationships: Equipment and product - The equipment "produced" the product. Product and order - The product "belongs to" the order. Equipment and failure - The equipment "needs maintenance".

[0187] Use a graph database (such as Neo4j) to store entity and relationship information and support dynamic updates.

[0188] 4.2 Knowledge Graph Query

[0189] Through the intelligent query module, operators can access the knowledge graph using natural language. For example:

[0190] Query: "What are the recent temperature anomaly records of smelting furnace B?"

[0191] Query result: "There were 3 temperature anomalies in smelting furnace B from November 1st to November 10th, 2023, with the highest value being 1300°C."

[0192] Step 5: Intelligent Query and Fault Warning

[0193] 5.1 Intelligent Query Module

[0194] The intelligent agent, based on a fine-tuned open-source large language model (LLM), understands the operator's query and generates corresponding knowledge graph query statements.

[0195] User input: "Which orders were delayed due to the failure of rolling mill A?"

[0196] System output: "Order #12345 and Order #56789 were delayed in delivery due to the failure of rolling mill A on November 15th, 2023."

[0197] 5.2 Fault Warning Module

[0198] Real-time monitor the data streams of equipment such as rolling mills and smelting furnaces, and trigger a warning if an anomaly is detected. The warning content includes:

[0199] Equipment name: Rolling mill A.

[0200] Anomaly time: November 20th, 2023, 14:35.

[0201] Anomaly parameter: Vibration frequency exceeds the standard (current value 45Hz, normal value range 10 - 30Hz).

[0202] Suggested measures: Check the bearings and arrange for maintenance.

[0203] The warning information is automatically sent to the maintenance department and relevant responsible persons through the management platform.

[0204] Step 6: Integration and Closed-loop Management

[0205] 6.1 Integration of Data Stream and Control Flow

[0206] All monitoring data is integrated into a unified platform, the "Intelligent Factory Management Center", where operators can view it in real time:

[0207] Data Dashboard: Displays the real-time trends of key indicators such as temperature and vibration.

[0208] Historical Records: Query the equipment operation status and product quality data for a specific time period.

[0209] Warning Information: Respond quickly to anomalies.

[0210] 6.2 Decision Support

[0211] The intelligent agent combines historical data with real-time analysis to provide optimization suggestions. For example:

[0212] Suggest adjusting the operating speed of the rolling mill to reduce vibration.

[0213] Prompt to increase the smelting time appropriately to improve product consistency.

[0214] Through the above steps, the stainless steel factory has achieved real-time perception and intelligent analysis of data in production, optimized production efficiency, reduced the impact of equipment failures, and improved the overall management level. This implementation method, while clarifying the process and division of responsibilities, empowers traditional manufacturing with advanced technologies, providing an example for the intelligent transformation of the factory.

[0215] It should be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, those of ordinary skill in the art should understand that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A data perception and process data modeling method for lean manufacturing in smart factories, characterized in that: The following steps are involved: Configure the data acquisition module to collect data in real time from sensors on the production line, machine vision systems for detecting product surface defects, PLC controllers, and ERP systems. The collected data is transmitted to the data acquisition server via industrial Ethernet. The data acquisition software running on the data acquisition server is responsible for parsing and storing the data. In the data fusion module, the collected data are cleaned, standardized, enhanced, aligned, and fused; A hybrid deep learning model of convolutional neural network (CNN) and long short-term memory (LSTM) is used to process the sorted real-time data, where CNN is used to process sensor data and machine vision data and extract spatial features; LSTM is used to process PLC data, ERP data, and sensor time series data to capture time dependencies and dynamic changes. The hybrid deep learning model outputs various triples for knowledge graph modeling, including entity, relationship, and attribute information. Based on the output of the deep learning model, a knowledge graph is constructed to clarify the relationship between entities; Use intelligent query modules to query knowledge graphs in real time and interact with users in natural language; A fault warning module is used to monitor data in real time, identify anomalies and generate warning information; the warning information includes the specific value of the abnormal data, timestamp and information of the production line that may be affected. The warning information is updated in real time through the knowledge graph and is automatically detected by the intelligent agent and sent to relevant personnel.

2. The data perception and process data modeling method for lean manufacturing in smart factories according to claim 1, characterized in that: Further including: Integrate all monitoring data into a unified platform to achieve closed-loop management of data flow and control flow.

3. The data perception and process data modeling method for lean manufacturing in smart factories according to claim 1, characterized in that: The cleaning, standardization, enhancement, alignment and fusion of the collected data includes: Use the median filter algorithm to handle the noise in the sensor data; perform data integrity checks on the order information in the ERP system to ensure that all necessary fields including order number, batch number and quantity are correctly filled in; For missing or abnormal data, reverse search is performed in the ERP system based on the timestamp and batch number to supplement the missing order number. If a negative or unreasonable value appears in the quantity field, correction is made based on the average or median of historical order data. The data collected by the PLC controller includes equipment operating status, control signals and operation logs. The time window-based interpolation method is used to fill in the missing values ​​of the time series in the equipment operating status data to ensure the continuity and integrity of the time series data. For the equipment operation logs recorded by the PLC, the key event extraction method is used to identify equipment start-stop, faults and changes in operating modes, and eliminate redundant and irrelevant log information.

4. The data perception and process data modeling method for lean manufacturing in smart factories according to claim 1, characterized in that: The standardization of cleaning, standardizing, enhancing, aligning and fusing the collected data includes: Sensor data standardization: converting parameters including temperature, pressure, and humidity into standard units; ERP system data standardization: perform field renaming and unit conversion on data including orders, inventory and process parameters; Machine vision data standardization: normalize the extracted image defect feature values ​​to the same dimensional range; PLC data standardization: Normalize the equipment operating status and control signal data, and standardize the timestamp.

5. The data perception and process data modeling method for lean manufacturing in smart factories according to claim 1, characterized in that: The knowledge graph constructed based on the output of the deep learning model includes: entity information, relationship information, multimodal attribute data, event sequence, spatial location information and reasoning auxiliary data.

6. The data perception and process data modeling method for lean manufacturing in smart factories according to claim 1, characterized in that: The real-time query of the knowledge graph using the intelligent query module and interacting with the user in the form of natural language includes: The natural language sentence input by the user is represented as a word sequence S = {w1,w1,w n The natural language understanding (NLU) module uses the fine-tuned LLM model to perform semantic analysis on the input and convert it into query intent I and slot information F, namely: I,F=f NLU (S) Among them, f NLU Represents the NLU function. The parsed intent I and slot F are input into the query generation module to generate a query statement Q for the knowledge graph: Q=f QueryGen (I,F) f QueryGen It is a query generation function used to map the parsing results into an executable query language; The generated query Q is executed on the knowledge graph KG, and the query result R is obtained: R=KG(Q) The knowledge graph KG consists of a series of triples (s i ,p i ,o i ),where s i is the head entity, p i It's a relationship, i It is the tail entity. The result processing and summary module sorts R and obtains the refined result R′: R′=f Process (R) f Process Functions are used for operations including data cleaning, information summarization, and result sorting; The processed result R' is input into the natural language generation (NLG) module to generate a user-readable natural language response S resp : S resp =f NLG (R′) f NLG The fine-tuned LLM model is used to convert structured data into fluent natural language.

Images