Glass processing dynamic monitoring method and system based on multi-source data fusion

Abstract

The invention provides a glass processing dynamic monitoring method and system based on multi-source data fusion, and relates to the field of industrial artificial intelligence, the method comprises the following steps: continuously collecting sound signals generated during processing, and carrying out noise reduction processing on the obtained sound signals to obtain sound data after noise reduction; continuously monitoring the acoustic data after noise reduction, acquiring abnormal acoustic characteristics of the glass in the processing process, determining an accurate time point of the abnormal acoustic characteristics, and acquiring time information of the abnormal acoustic characteristics; acquiring a glass processing position at the time of the abnormal acoustic features according to the time information of the abnormal acoustic features, and acquiring image or video information of a glass processing area within the time range; according to the obtained abnormal acoustic characteristics and the image or video information of the glass processing area, whether defect information appears in the glass processing process or not is judged, so that real-time dynamic monitoring and defect accurate positioning can be realized in the glass processing process, and specific process parameter adjustment suggestions are generated.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial artificial intelligence, and particularly relates to a dynamic monitoring method and system for glass processing based on multi-source data fusion. Background Art

[0002] Quality control in glass processing is a crucial step to ensure that products meet safety, durability, and aesthetic standards. Traditional methods usually inspect the glass after processing to identify defects and analyze the causes.

[0003] The methods mainly include using industrial cameras or manual visual inspection to take static photos or magnifying glass inspections of the surface of the finished product, counting defects such as cracks, chipping, and edge asymmetry and their quantities, and evaluating the quality accordingly. However, since this method can only be executed after processing, it cannot capture real-time defect information during the processing. And these data need to be retrieved by professional staff through image information from the production logs of the glass, and the data that cause defects are extracted from the massive data of the production logs to analyze which data are the ones that lead to defects, and based on this, seek parameter combinations to reduce the defect rate. But this method has a long process cycle (from several hours to several days), a lag in feedback, a low accuracy rate in judging the cause of defects, and it is difficult to respond to the high-tempo production requirements in a timely manner.

[0004] It can be seen that the existing technology mainly relies on offline vision and manual inspection after processing, and has problems such as poor real-time performance, difficulty in defect traceability, low accuracy, and long feedback cycles. This method cannot capture and locate defects in a timely manner during the processing, and it is also difficult to quickly and accurately judge the cause of defects, thus affecting the timely adjustment of process parameters and the improvement of product quality. Summary of the Invention

[0005] This application aims to solve the technical problems in the prior art that during the glass processing, defects generated during the processing cannot be collected and monitored in real time, relying on the experience of staff for judgment, and it is difficult to trace the source of defects, resulting in a low accuracy rate in judging the cause of defects, a long cycle for defect analysis and parameter adjustment, and the inability to quickly respond to abnormal situations during the processing, leading to low production efficiency and a high scrap rate. The following technical solutions are designed, and the specific content is as follows.

[0006] A dynamic monitoring method for glass processing based on multi-source data fusion, the method comprising: Continuously collect the sound signals generated during processing, and perform noise reduction processing on the obtained sound signals to obtain the noise-reduced sound data; Continuously monitor the noise-reduced acoustic data, obtain the abnormal acoustic characteristics that occur during the glass processing, and determine the precise time point of the abnormal acoustic characteristics to obtain the time information of the abnormal acoustic characteristics; Obtain the glass processing position at the time of the abnormal acoustic feature through the time information of the abnormal acoustic feature, and obtain the image or video information of the glass processing area within this time range; Judge whether there are processing defects during the glass processing according to the obtained abnormal acoustic feature and the image or video information of the glass processing area.

[0007] Among them, the method for judging whether there are processing defects during the glass processing is as follows. After preprocessing the obtained image information, apply an automated image processing algorithm to detect whether there are abnormal features on the glass surface; If the image processing module detects obvious abnormal features in the glass processing area, classify the detected abnormal features to obtain the processing defect information of the glass; If the image processing module does not detect obvious abnormal features in the glass processing area, it is determined that there are no processing defects.

[0008] If there are processing defects, extract the image and other processing feature information within the corresponding time period through the time information of the abnormal acoustic feature; Fuse all the feature information, analyze the defect type, and obtain the defect classification result; Combined with the defect classification result, analyze the cause of the processing defect; Based on the cause of the defect, give specific suggestions for adjusting the process parameters.

[0009] During the development of the embodiments of the present application, the inventors found that the prior art mainly relies on visual or manual inspection of the finished product after glass processing to judge whether there are processing defects, and then retrieve the production log and compare the data to analyze the cause of the defect and give a subsequent process adjustment plan. However, this method has obvious hysteresis, is time-consuming and laborious, and has poor results, and it is difficult to meet the requirements of efficient and real-time production quality control.

[0010] To overcome these problems, the inventors proposed a monitoring scheme based on real-time image acquisition. By collecting and online analyzing the image information during the glass processing, once a processing defect is found, the location of the defect can be immediately located, and the production data within the corresponding time period can be used for cause analysis. However, this method will cause a large amount of image data to be processed simultaneously, resulting in waste of computing power and a significant increase in cost.

[0011] In the process of further in-depth research, the inventors noticed that during the glass processing, the sound emitted by the equipment basically remained in a relatively stable state. When processing defects occurred, the corresponding acoustic signals would show obvious fluctuations. Based on this discovery, the inventors conceived to use sound monitoring as a defect trigger mechanism, that is, when abnormal acoustic fluctuations were detected, only the image information at that moment was collected for defect determination, and then the vibration, temperature, and mechanical data during that time period were obtained for cause analysis and archiving. This not only can quickly and accurately determine the defect occurrence time, processing location, and defect type, but also greatly improves the efficiency of defect analysis, thereby enhancing the yield of glass processing.

[0012] In addition, the solution of this application also adopts the strategy of separately configuring directional acoustic sensors on each production line to ensure that each sensor mainly collects the sound signals in the target cutting machine area. Combined with noise reduction processing, unified time stamping, and frame segmentation technology, abnormal acoustic events corresponding to specific cutting machines are effectively extracted from the overall noise, avoiding interference from abnormal signals on other production lines. Even if abnormal acoustic signals from other production lines are occasionally collected and no defects are found through image analysis, it will not have a negative impact on the overall defect detection accuracy (this accuracy refers to the probability of detecting all processing defects on a piece of glass, rather than simply based on whether there are defects in the images collected according to abnormal acoustic signals).

[0013] Among them, the other processing feature information includes vibration data, temperature data during the glass processing, and mechanical data during the glass processing.

[0014] Furthermore, the original data of the acoustics, vibration, temperature, and mechanics are denoised, filtered, and time synchronized to obtain the denoised data of each channel; Generate a unified time stamp for the denoised data of each channel to obtain a multi-source data stream with time alignment, so that the acoustic, image, vibration, temperature, and mechanical data can form a logical association at the same moment.

[0015] Furthermore, the method of fusing all the feature information and analyzing the defect type to obtain the cause of the processing defect is as follows: After standardizing the acoustic (including abnormal voiceprints), image, vibration, temperature, and mechanical features, generate a comprehensive feature vector through an attention mechanism or other fusion algorithms; Use a trained multi-modal model (such as LSTM, CNN-LSTM, etc.) to perform state prediction and defect classification on the fused features, and output the current defect type and its confidence level; Combined with the defect classification results, further use decision trees, Bayesian networks, or regression models to analyze the causes of defects (such as too high temperature, uneven stress, abnormal vibration, etc.) to generate processing defect information.

[0016] Further, the method of the present application further includes: By analyzing the processing defect information, specific suggestions for adjusting the process parameters are given; According to the suggestions for adjusting the process parameters, a feedback instruction is automatically generated and sent to the processing equipment; All the detection, analysis, and feedback process data are stored to construct a defect feature library and a historical data database; The abnormal acoustic features and their corresponding defect features are entered into the defect feature library, so that the defect type corresponding to the abnormal acoustic features can be quickly obtained through voiceprint matching.

[0017] In summary, the method provided by the present application realizes the real-time detection, precise positioning, accurate classification, and timely feedback of defects in the glass processing process by combining real-time acoustic monitoring, directional image acquisition, and multi-source data fusion analysis, greatly improving the processing efficiency and product quality, while reducing the production cost and the risk of manual intervention.

[0018] On the other hand, the present application provides a dynamic monitoring system for glass processing based on multi-source data fusion; Data acquisition module: used to continuously and real-time collect acoustic, image, vibration, temperature, and mechanical data; Data preprocessing and synchronization module: perform noise reduction, filtering, and strict time alignment on all the collected data; Abnormal trigger and feature extraction module: use the acoustic data for real-time monitoring to trigger abnormal detection; then combine the image data analysis to check whether there are processing defects. If there are processing defects, the vibration, temperature, and mechanical features are extracted; at the same time, the defect type can also be identified by performing voiceprint matching; Multi-source data fusion and defect cause analysis module: after fusing all the extracted features, perform state prediction and defect classification through an intelligent model, and then use the decision model to analyze the defect cause and generate specific suggestions for adjusting the process parameters; Real-time monitoring and feedback control module: used to display the defect information and analysis results on the visualization interface in real-time, automatically generate feedback control instructions and send them to the equipment, and record all the historical data at the same time, supporting online learning and continuous optimization.

[0019] To solve the above technical problems, one or more technical solutions provided in the present application have at least the following technical effects or advantages: The present application can capture defect information in real-time during the glass processing process and precisely locate the time, location, and type of the defect occurrence; The present application uses multi-source data fusion and deep models to realize the accurate identification of defect types, enabling it to automatically and quickly infer the root cause of the defect generation; This application can generate and issue adjustment suggestions in real time, reduce the defect rate through automatic feedback control, and improve production efficiency and product quality; The above description is only an overview of the technical solution of this application. In order to be able to understand the technical means of this application more clearly, it can be implemented in accordance with the content of the specification. And in order to make the above and other purposes, features and advantages of this application more obvious and understandable, the specific implementation manners of this application are specifically given below. Description of the Drawings

[0020] Figure 1 It is a schematic flow chart of a glass processing dynamic monitoring method based on multi-source data fusion provided by an embodiment of this application.

[0021] Figure 2 It is a schematic flow chart of step S300 of a glass processing dynamic monitoring method based on multi-source data fusion provided by an embodiment of this application.

[0022] Figure 3 It is a schematic structural diagram of a glass processing dynamic monitoring system based on multi-source data fusion provided by an embodiment of this application.

[0023] Description of the reference numerals: Data acquisition module 10, data preprocessing and synchronization module 20, anomaly trigger and feature extraction module 30, multi-source data fusion and defect cause analysis module 40, real-time monitoring and feedback control module 50. Specific Embodiments

[0024] An embodiment of this application provides a glass processing dynamic monitoring method based on multi-source data fusion. By using multi-source data fusion technology, it continuously collects sound, image, vibration, temperature and mechanical data during the glass processing process in real time, and through multi-level noise reduction, filtering and time synchronization processing, it ensures that the data of each channel is accurately aligned on the same time scale. Subsequently, an acoustic anomaly trigger mechanism is used to determine the time of abnormal acoustic features, and the corresponding images and other physical features are extracted during this period. Through the attention mechanism and deep time series models (such as CNN-LSTM), efficient fusion and defect classification of multi-modal features are realized. At the same time, methods such as decision trees and Bayesian networks are used for root cause analysis. Finally, feedback control instructions are automatically generated according to the analysis results to achieve closed-loop process optimization; Through the above technical means, this solution can achieve real-time dynamic monitoring and precise defect positioning during the glass processing process, effectively identify and classify various defects in the processing, automatically infer the causes of defects, generate specific process parameter adjustment suggestions, and achieve automatic closed-loop feedback control. This system significantly shortens the response cycle of defect detection and process adjustment, improves the processing quality and production efficiency, and at the same time provides data support for online learning and optimization for continuous improvement; After introducing the basic principle of the present application, various non-limiting implementation manners of the present application will be specifically introduced below in conjunction with the accompanying drawings of the specification. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0025] Embodiment 1: As Figure 1 shown, A dynamic monitoring method for glass processing based on multi-source data fusion: S100 Continuously collect the sound signals generated during processing, and perform noise reduction processing on the obtained sound signals to obtain the noise-reduced sound data; Continuously monitor the noise-reduced acoustic data, obtain the abnormal acoustic characteristics that occur during the glass processing, and determine the exact time point of the abnormal acoustic characteristics, and obtain the time information of the abnormal acoustic characteristics, so as to accurately determine the time of the abnormal acoustic characteristics when an abnormal acoustic event occurs, and provide a trigger basis for subsequent defect location and multi-source data fusion analysis; Specifically, it is by installing a high-sensitivity microphone or acoustic emission sensor on the glass processing equipment, and continuously sampling the original sound signals generated during the glass processing through a collection device (such as a built-in ADC). The sensor is usually set at a position in close contact with the processing part to capture as much as possible the acoustic wave changes caused by physical phenomena such as microcrack propagation or fracture; Before the signal enters the analog-to-digital conversion, it is first preliminarily processed by the front-end analog circuit. Specifically, it includes using an analog band-pass filter, which can limit the signal collection to only a specific frequency band (for example, the frequency range related to material rupture), and amplify the weaker signal through a pre-amplifier to improve the signal-to-noise ratio; After converting the analog signal into a digital signal, various digital filtering technologies are used to further process the sound data. Common methods include: Using a digital band-pass filter to perform secondary filtering on the signal to remove low-frequency environmental noise and high-frequency interference.

[0026] Adopting a wavelet denoising algorithm to decompose the signal into multiple scales, removing the noise components according to the threshold at each scale, and then performing signal reconstruction to make the result smoother and retain the acoustic characteristics related to the defects.

[0027] Using adaptive filtering (such as Kalman filtering) to track the dynamic signal to further remove random noise and environmental interference; For the noise-reduced acoustic data, the processing system will embed accurate timestamps in the data stream to obtain high-quality sound data that has been noise-reduced, filtered, and time-stamped to ensure that the subsequent detection and comparison of abnormal events can accurately correspond to the actual occurrence time; Perform real-time analysis on the denoised acoustic data to identify abnormal acoustic features, enabling real-time analysis based on the denoised data using methods such as frame segmentation, energy, and peak analysis to accurately capture abnormal sounds and determine the precise time point of the abnormal acoustic features, and transmit this time point as a defect trigger signal to subsequent modules; Specifically, the continuous denoised sound data is frame-segmented at fixed time intervals (for example, 1 - 5 milliseconds as a time window). Statistical analysis is performed on the data within each time window, such as calculating the total energy within each window, taking the maximum amplitude value, and the envelope change; Methods such as short-time energy detection, peak detection, and signal envelope analysis are used to capture sudden changes; Set a dynamic threshold so that when the short-time energy or peak of a certain frame or consecutive frames significantly exceeds the preset background level, it can be preliminarily determined that there is an abnormal acoustic event. For example, a significant sudden increase in the usually stable noise level usually means that the material has cracks or fractures; When an abnormality in the acoustic signal (such as a sharp increase in energy or a sudden change in peak) is detected and after multiple statistical judgments (such as simultaneously meeting multiple characteristic indicators), the recorded time stamp of the abnormal trigger event and the corresponding description of the acoustic characteristics are simultaneously transmitted as a trigger signal to the downstream module, which is used to guide the subsequent module to extract image, vibration, temperature, and mechanical data within a specific time window for comprehensive analysis.

[0028] S200 obtains the glass processing position of the abnormal acoustic feature time based on the time information of the abnormal acoustic feature, and obtains image or video information of the glass processing area within this time range; Judge whether there are processing defects in the glass processing process based on the obtained abnormal acoustic characteristics and the image or video information of the glass processing area.

[0029] Specifically, the method for judging whether there are processing defects in the glass processing process is as follows. After preprocessing the obtained image information, an automated image processing algorithm is applied to detect whether there are abnormal features on the glass surface; If the image processing module detects obvious abnormal features in the glass processing area, classify the detected abnormal features to obtain the processing defect information of the glass; If the image processing module does not detect obvious abnormal features in the glass processing area, it is considered that there are no processing defects.

[0030] Specifically, by using the abnormal trigger time information recorded in S100, the corresponding image frame is accurately located in the continuously acquired video stream, thereby ensuring strict time matching between the image data and the acoustic abnormal event; That is, it can mark continuous timestamps for image data during data acquisition. After outputting the exact time point triggered by a defect through S100, immediately use this time marker to search for the corresponding image frame in the video data stream. By comparing the timestamps, ensure that the selected frame exactly corresponds to the moment of the abnormal acoustic feature; Therefore, once the time of the abnormal acoustic feature is determined, the image frame at this moment will be extracted from the real-time buffer of the video stream. To fully record the processing dynamics before and after the defect, several frames before and after the trigger time point can also be extracted. These frames will provide continuous processing state information, which helps subsequent defect location and cause analysis.

[0031] Through the above method, during the entire image extraction process, it can ensure that the time information of the image frame is completely consistent with the abnormal event, avoiding information confusion caused by time deviation, thereby providing accurate visual data support for subsequent preprocessing; The step S200 further includes the following steps: Preprocess the image frame obtained in step S200, extract the feature information related to the defect in the processing area, make the image data clearer and the structure more explicit, so as to facilitate subsequent defect analysis; The specific steps are as follows: First, perform lens distortion correction on the extracted image frame, and use the previously obtained camera calibration parameters to correct the image distortion caused by lens characteristics, ensuring that the image size and shape truly reflect the state of the processing area, so as to preprocess its image information; According to the predefined glass processing area or use the on-site positioning algorithm to automatically identify the region of interest (ROI), and crop the image. This step makes the subsequent processing only focus on the area where the defect may appear, reduces irrelevant information, and reduces the data processing load; Apply an edge detection algorithm (such as the Canny algorithm) to the cropped image to extract the obvious edge features in the image, thereby highlighting possible cracks, chipping or surface discontinuity areas. These edge information helps to intuitively judge the position and shape of the defect and provides the basis for subsequent quantitative analysis; After the finally obtained image data undergoes distortion correction, ROI cropping and edge feature extraction, it will become clearer and more focused on the defect area, and become an important visual input for subsequent multi-modal data fusion and state prediction; That is, this step realizes the extraction of the image corresponding to the defect event by obtaining the defect trigger time and precisely aligning it with the image frame in the video stream; subsequently, preprocess the extracted image, such as distortion correction, region cropping and edge detection, so as to obtain a clear defect image and the visible features of the processing area. Through the completion of these two steps in step S200, it provides accurate and intuitive visual data support for subsequent multi-modal feature fusion and defect cause analysis.

[0032] If there are processing defects in S300, the images and other processing feature information within the corresponding time period are extracted through the time information of abnormal acoustic features; All the feature information is fused, the defect types are analyzed, and the causes of processing defects are obtained; Based on the causes of the defects, specific process parameter adjustment suggestions are given.

[0033] This step S300 includes the following steps: If there are processing defects in S310, the images and other processing feature information within the corresponding time period are extracted through the time information of abnormal acoustic features, and the other processing feature information includes vibration data, temperature data during the glass processing, and mechanical data during the glass processing; That is, after receiving the acoustic anomaly trigger signal, data is immediately extracted from the sensors installed at key positions of the processing equipment within the defined time window, mainly including: Vibration data, the vibration fluctuation data collected by the vibration sensor, and the vibration data is band-pass filtered to retain the key frequency bands; Temperature data, the temperature change data collected by the thermocouple or infrared sensor, and the temperature data is processed through low-pass filtering; Mechanical data, the processing load or force information collected by the force sensor, and the steady-state trend separation and fluctuation analysis are performed on the mechanical data; It can ensure that the data sources are unified with the defect trigger time as the reference point. Through timestamp reconstruction, the data of all channels are aligned to the same time axis, ensuring high synchronization of multi-source data in time, forming a multi-dimensional feature basis with consistent time series, so that the acoustic, image, vibration, temperature, and mechanical data can form a logical association at the same moment.

[0034] S320 fuses all the feature information, analyzes the defect types, and obtains the defect classification results; Among them, the method of fusing all the feature information, analyzing the defect types, and obtaining the causes of processing defects is as follows: After standardizing the acoustic (including abnormal voiceprint), image, vibration, temperature, and mechanical features, a comprehensive feature vector is generated through the attention mechanism or other fusion algorithms; Specifically: Since the data of different types of sensors have large differences in numerical scales due to different physical quantities (such as sound in decibels, temperature in degrees Celsius, vibration in acceleration, etc.). Therefore, it is necessary to perform unified normalization or standardization processing on each type of data to make it fall within the same numerical range, thereby enhancing the comparability and fusion of the data.

[0035] That is, representative feature variables are extracted from each channel, and the method is as follows: Extract the voiceprint envelope feature from the sound data; Extract the number of edges, contour integrity, etc. from the image data; Extract the spectral energy distribution from the vibration data; Extract the gradient change or abnormal fluctuation amplitude, etc. from the temperature and mechanical data.

[0036] Adopt a deep fusion network structure (such as using CNN to process image features, LSTM to process time series features, and then connecting and fusing them) to form the final comprehensive feature vector.

[0037] Use a trained multi-modal model (such as LSTM, CNN-LSTM, etc.) to perform state prediction and defect classification on the fused features, and output the current defect type and its confidence level; That is, by calling a pre-trained multi-modal classification model (such as CNN-LSTM, Transformer, or a fused neural network), and inputting the comprehensive feature vector of the current defect event into the model; The model recognizes the input data according to the feature patterns it has learned, and determines whether the current state belongs to a certain type of known defect, such as: edge chipping; surface scratches, cracks, etc. defects, and obtains the defect classification result.

[0038] Combined with the defect classification result, S330 further uses a decision tree, Bayesian network, or regression model to analyze the causes of defects (such as too high temperature, uneven stress, abnormal vibration, etc.) to generate processing defect information; That is, after the defect event occurs, the system traces back various process parameters and sensor data during this period, including temperature, vibration, mechanical load, spindle speed, feed speed, etc. By comparing the data patterns during abnormal occurrence with those during normal processing, analyze the change trends and fluctuation amplitudes of each parameter; Then, based on the existing knowledge model or historical data samples in the system, analyze using the following methods, for example: Decision tree method, establish the determination path between different defect types and parameter changes; Bayesian network, evaluate the probability impact of each parameter abnormality on the occurrence of defects; Regression analysis, measure the linear or non-linear relationship between the changes of each parameter and the defect degree; According to the above analysis, the system can obtain the main inducements of defects. For example, if the temperature curve rises rapidly when the defect occurs, it is judged as insufficient cooling or heat accumulation; if the stress data suddenly fluctuates violently, it is determined as unstable feed or uneven stress; if abnormal frequency bands appear in the vibration signal, it is considered caused by mechanical component instability or wear; obtain the cause analysis result.

[0039] Based on the causes of defects, S400 gives specific suggestions for adjusting process parameters, and automatically generates control instructions to be fed back to the control system to achieve the purpose of closed-loop defect suppression; That is: the system matches corresponding coping strategies for the identified causes. For example: For too high temperature, it is recommended to increase the coolant flow rate or shorten the continuous processing time; If uneven force is found, it is recommended to reduce the feed speed and increase the clamping stability; If defects are caused by excessive vibration, it is recommended to reduce the cutting depth or check the tool status; And based on the suggestions, the parameter adjustment amount is converted into a standardized control instruction. For example: Increase the cooling pump control signal by 10%; Reduce the feed rate from 120 mm / min to 100 mm / min; Start the automatic tool detection subroutine to detect the tool wear condition; The system sends the control instruction to the controller of the processing equipment (such as PLC, CNC system, etc.) to achieve automatic parameter correction. This process can be: semi-automatic mode (confirmed and executed after operator review) or full-automatic closed-loop (the system executes the instruction itself and quickly adjusts the processing state).

[0040] After the control strategy is executed, the system continuously monitors whether the new round of data improves the defect performance. If the defect still exists, the system will conduct a new causal analysis and strategy fine-tuning to achieve continuous closed-loop optimization; S500 stores all the process data of detection, analysis and feedback, and constructs a defect feature library and a historical data database; Enter the abnormal acoustic features and their corresponding defect features into the defect feature library, so that the defect type corresponding to the abnormal acoustic features can be quickly obtained through voiceprint matching.

[0041] Specifically, during the closed-loop operation of the entire glass processing monitoring, the system automatically archives the original data, preprocessed data, feature information generated at each stage, defect classification results, and issued control instructions collected, forming a complete historical data database. This step ensures that there is a reliable data basis for subsequent statistical analysis, model update and fault tracing; Specifically, the system classifies and organizes the data from different modules (such as acquisition, preprocessing, abnormal trigger, feature fusion and defect classification, feedback control) according to types. For example: Original data: including original sensing data such as acoustics, images, vibrations, temperatures, mechanics, etc.

[0042] Preprocessing results: Data after noise reduction, filtering, and normalization.

[0043] Feature information: Various feature vectors extracted from each channel and the integrated features after fusion.

[0044] Defect classification and status prediction data: Defect types, confidence levels, abnormal trigger times, etc. output by the model.

[0045] Control instructions and feedback: Parameter adjustment instructions automatically generated and issued, and their execution confirmation information; And a database system is used to store data. The system will store time-series data in a time-series database (such as InfluxDB or OSIsoft PI), and store event records, defect labels, and control instructions in a relational database (such as PostgreSQL or MySQL); It can automatically package and archive data in different time periods (such as by shift, by day) through circular buffering and regular archiving; For large-capacity real-time acquisition data, data compression technology can also be used to ensure that long-term storage does not occupy too much space, while allowing efficient retrieval and backtracking; So that a complete and structured glass processing historical database can be constructed, which records data at each stage, detection results, and feedback instructions, facilitating subsequent statistics, analysis, and model updates.

[0046] Establish a comprehensive defect feature library, and enter the defect-related features detected during the production process (including voiceprint features, image features, and features extracted by other sensors) into the system to support the rapid identification and traceability of similar defects in the future; That is, the system automatically extracts multi-modal feature data after standardization processing, and associates and labels it with the corresponding defect classification results (such as "micro-cracks", "chipping", etc.); For each defect event, record the main feature data: Acoustic features: Abnormal voiceprint information (such as short-time energy, envelope features).

[0047] Image features: Edge, shape, and area information of the defect area.

[0048] Vibration, temperature, and mechanical features: Specific parameter change data related to defects.

[0049] Enter all the labeled data to build a database or knowledge base, and each record contains defect type, key features, occurrence time, and process background information; The knowledge base supports retrieval based on time, defect type, and feature similarity, facilitating future real-time comparison and traceability; Use the acoustic features entered into the knowledge base as templates to establish a voiceprint matching algorithm: When the system detects a new abnormal acoustic signal, it can compare it with the existing voiceprint templates in the feature library and calculate the similarity; When the similarity exceeds the set threshold, the system can quickly determine the matching relationship between the newly detected defect and the historical defect, further guiding fault tracing and parameter adjustment; Moreover, the system can regularly update the feature library automatically, summarize the characteristics of the latest-occurring defects, and re-label and adjust the weights of the old records according to the feedback results; And it supports manual review and supplementation to ensure that the feature library always reflects the latest process conditions and defect patterns On the other hand, the present application provides a glass processing dynamic monitoring system based on multi-source data fusion, and this system includes; Data acquisition module 10: used for continuously and real-time collecting acoustic, image, vibration, temperature and mechanical data; The data acquisition module 10 includes: The acoustic data acquisition unit is used to obtain all the sound waves and acoustic emission signals generated during the processing, output the original sound data, and attach an accurate timestamp to each data point; this data is used to detect the instantaneous abnormality during microcrack or fracture, and determine the abnormal acoustic feature time.

[0050] The image / video data acquisition unit is used to obtain the image or video stream of the processing site in real time, and when an abnormal acoustic signal is detected, the system quickly extracts the image frame at that moment according to the recorded time point, for positioning the defect area and the glass processing progress.

[0051] The vibration data acquisition unit is used to obtain the vibration information of the mechanical equipment and the glass, including data such as amplitude and frequency distribution.

[0052] The temperature data acquisition unit is used to obtain the temperature change information of the contact area between the glass and the tool, upload the temperature data in real time, and record the detailed changes, facilitating subsequent analysis of whether the thermal abnormality is related to the defect.

[0053] The mechanical data acquisition unit is used to monitor the force condition when the tool contacts the glass, including the pressure and the change trend of the force, obtain the original mechanical data, and attach the acquisition time information, providing a basis for subsequent judgment of defects caused by abnormal force.

[0054] Data preprocessing and synchronization module 20: perform noise reduction, filtering and strict time alignment on all the collected data: The data preprocessing and synchronization module 20 includes: The signal filtering and noise reduction unit is used to process the original acoustic, vibration, temperature and mechanical data collected by the data acquisition module 10, so as to output high-quality data after noise reduction and smoothing, ensuring that key information is retained while the noise is greatly reduced.

[0055] Data synchronization and time alignment unit, which uses a phase-locked loop (PLL) or an embedded synchronization module to generate a unified timestamp for each channel of data after noise reduction, ensuring that the data streams are precisely matched on the time axis to obtain a multi-source data stream that is time-aligned, so that acoustic, image, vibration, temperature, and mechanical data can form a logical association at the same moment.

[0056] Abnormal trigger and feature extraction module 30: Use real-time monitoring of acoustic data to trigger abnormal detection; then combine image data analysis to check whether there are processing defects. If there are processing defects, extract vibration, temperature, and mechanical features; at the same time, voiceprint matching can also be performed to identify the defect type; The abnormal trigger and feature extraction module 30 includes; Real-time acoustic monitoring and abnormal trigger unit: Set an abnormal threshold by using spectrum analysis and energy detection, and monitor the change of acoustic data in real time. Once abnormal sounds (such as a sharp rise in energy, a peak mutation) are detected, immediately record the accurate time point of the abnormal acoustic features, and output the timestamp and corresponding features of the abnormal acoustic event for triggering subsequent processing.

[0057] Image / video information extraction and defect recognition unit, which is used to extract the corresponding image frames from the video according to the obtained abnormal time points, and use image processing techniques (such as edge detection and texture analysis) to judge whether there are processing defects. If there are processing defects, determine the area where the defects are located to obtain the image features at the defect moment and the intuitive position information of the processing site.

[0058] Vibration, temperature, and mechanical feature extraction unit, which is used to extract the amplitude and frequency distribution in the vibration data, the rising rate and peak value in the temperature data, and the force peak value and change rate in the mechanical data within the abnormal time window after confirming the existence of processing defects, so as to obtain multi-dimensional physical features within the abnormal time window for describing the dynamic state of the processing equipment and the glass at the moment of defect occurrence.

[0059] Voiceprint matching unit, which is used to compare the acoustic features obtained at the abnormal moment with the pre-established defect feature library, and quickly confirm the defect type corresponding to the current abnormal acoustic signal through the "voiceprint matching" algorithm to obtain the defect type label and the matching confidence level, as an important reference for subsequent defect analysis. This unit can be used when the defect feature library tends to be saturated after the long-term operation of this system, so that it can further reduce the operation cost of the system.

[0060] Multi-source data fusion and defect cause analysis module 40: After fusing all the extracted features, perform state prediction and defect classification through an intelligent model, and then use a decision model to analyze the defect causes and generate specific process parameter adjustment suggestions; The multi-source data fusion and defect cause analysis module 40 includes: A feature fusion unit, which is used to normalize and standardize the feature data of acoustics, images, vibrations, temperature, and mechanics, and then use the attention mechanism or multi-modal fusion algorithm to integrate various features into a comprehensive feature vector, so as to generate a vector comprehensively describing the current processing state and defect features, providing a basis for state prediction.

[0061] A state prediction and defect classification unit, which is used to adopt a pre-trained multi-modal model (such as LSTM or CNN-LSTM) for the fused feature vector to predict the current glass processing state in real time, output the defect type and its confidence level, so as to obtain the processing state prediction result and indicate whether there is an abnormality and defect classification (such as "micro-crack", "fracture", "overheat", etc.).

[0062] A defect cause analysis and parameter suggestion unit, which is used to deeply analyze the cause of the defect by using a decision tree, Bayesian network or regression model for the fused state information and defect classification result, identify the main influencing factors (such as abnormal temperature, unstable equipment vibration or uneven force), so as to generate a detailed defect cause report and corresponding process parameter adjustment suggestions, such as changing the feed speed, tool pressure or cooling measures.

[0063] The real-time monitoring and feedback control module 50: It is used to display the defect information and analysis results on the visualization interface in real time, automatically generate feedback control instructions and send them to the equipment, and record all historical data at the same time, supporting online learning and continuous optimization.

[0064] The real-time monitoring and feedback control module 50 includes: A real-time monitoring and display unit, which is used to display the state prediction, defect classification, defect cause analysis results and their image information through an industrial PC or a human-machine interface (HMI) in real time, showing the abnormal time, defect area, defect type, confidence level and cause report, and presenting the dynamic trend chart of each sensor at the same time, so as to obtain a visualization monitoring interface for the operator to quickly understand the current processing state and abnormal situation and record it as historical data.

[0065] An automatic feedback control unit, which is used to convert the generated process parameter adjustment suggestions into specific control instructions, such as adjusting the feed speed, tool pressure or coolant flow rate, and send them to the processing equipment in real time through a PLC or an embedded control system to achieve closed-loop feedback regulation, and obtain the actually adjusted processing parameters and feedback status, so as to reduce the probability of defect occurrence.

[0066] A data storage and online learning unit is used to obtain all the data generated by glass processing (including the processing results and feedback instructions at each stage), archive and store these data according to time, construct a defect feature library and a historical database, and support the online learning of subsequent models and the continuous optimization of feedback control strategies.

[0067] Through the detailed description of a glass processing dynamic monitoring method and system based on multi-source data fusion in this specification, those skilled in the art can clearly know a glass processing dynamic monitoring system based on multi-source data fusion in this embodiment. Since it corresponds to the method disclosed in the embodiment, the description is relatively simple. For related parts, refer to the description in the method section.

[0068] Through the above description of the disclosed embodiments, it is believed that those skilled in the art can implement or use this application. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dynamic monitoring method for glass processing based on multi-source data fusion, characterized in that The method includes: Continuously collect the sound signals generated during processing, and perform noise reduction processing on the obtained sound signals to obtain the noise-reduced sound data; Continuously monitor the noise-reduced acoustic data, obtain the abnormal acoustic characteristics that occur during the glass processing, determine the precise time point of the abnormal acoustic characteristics, and obtain the time information of the abnormal acoustic characteristics; Obtain the glass processing position at the time of the abnormal acoustic characteristics through the time information of the abnormal acoustic characteristics, and obtain the image or video information of the glass processing area within this time range; Judge whether there are processing defects during the glass processing according to the obtained abnormal acoustic characteristics and the image or video information of the glass processing area.

2. The dynamic monitoring method for glass processing based on multi-source data fusion according to claim 1, wherein The method for judging whether there are processing defects during the glass processing according to the obtained abnormal acoustic characteristics and the image or video information of the glass processing area is as follows: After preprocessing the obtained image information, apply an automated image processing algorithm to detect whether there are abnormal characteristics on the glass surface; If the image processing module detects obvious abnormal characteristics in the glass processing area, classify the detected abnormal characteristics to obtain the processing defect information of the glass; If the image processing module does not detect obvious abnormal characteristics in the glass processing area, it is considered that there are no processing defects.

3. The dynamic monitoring method for glass processing based on multi-source data fusion according to claim 2, wherein The method includes: If there are processing defects, extract the image and other processing feature information within the corresponding time period through the time information of the abnormal acoustic characteristics; Fuse all the feature information, analyze the defect types, and obtain the defect classification results; Combine the defect classification results and analyze the causes of the processing defects; Based on the causes of the defects, give specific suggestions for adjusting the process parameters.

4. A dynamic monitoring method for glass processing based on multi-source data fusion according to claim 3, characterized in that, The said other processing feature information includes vibration data, temperature data during the glass processing, and mechanical data during the glass processing.

5. The dynamic monitoring method for glass processing based on multi-source data fusion according to claim 4, characterized in that, Perform noise reduction, filtering, and time synchronization on the original acoustic, vibration, temperature, and mechanical data to obtain the noise-reduced data for each channel; Generate a unified time stamp for the noise-reduced data of each channel to obtain a multi-source data stream with time alignment, so that the acoustic, image, vibration, temperature, and mechanical data can form a logical association at the same moment.

6. The method for dynamically monitoring glass processing based on multi-source data fusion according to claim 5, wherein, The method for fusing all the feature information, analyzing the defect types, and obtaining the causes of the processing defects is as follows: Standardize the acoustic, image, vibration, temperature, and mechanical characteristics and generate a comprehensive feature vector; Use the trained multi-modal model to perform state prediction and defect classification on the fused features, and output the current defect type and its confidence level; Combine the defect classification results, and further use decision trees, Bayesian networks, or regression models to analyze the causes of the defects to generate processing defect information.

7. A dynamic monitoring method for glass processing based on multi-source data fusion according to claim 3, characterized in that Give specific suggestions for adjusting the process parameters by analyzing the processing defect information; Automatically generate a feedback instruction according to the process parameter adjustment suggestion and send it to the processing equipment.

8. A dynamic monitoring method for glass processing based on multi-source data fusion according to claim 7, characterized in that Store all the process data of detection, analysis, and feedback to build a defect feature library and a historical data database.

9. A dynamic monitoring method for glass processing based on multi-source data fusion according to claim 8, characterized in that The method includes: Enter the abnormal acoustic features and their corresponding defect features into the defect feature library, so that the defect type corresponding to the abnormal acoustic features can be quickly obtained through voiceprint matching.

10. A dynamic monitoring system for glass processing based on multi-source data fusion, characterized in that, The system is used to implement a dynamic monitoring method for glass processing based on multi-source data fusion according to any one of claims 1-9. The system includes: Data acquisition module: used to continuously and real-time collect acoustic, image, vibration, temperature, and mechanical data; Data preprocessing and synchronization module: perform noise reduction, filtering, and strict time alignment on all the collected data; Abnormal trigger and feature extraction module: use acoustic data for real-time monitoring to trigger abnormal detection; then combine image data analysis to check whether there are processing defects. If there are processing defects, extract vibration, temperature, and mechanical features; Multi-source data fusion and defect cause analysis module: after fusing all the extracted features, perform state prediction and defect classification through an intelligent model, and then use a decision model to analyze the defect cause and generate specific process parameter adjustment suggestions; Real-time monitoring and feedback control module: used to display the defect information and analysis results on the visualization interface in real time, automatically generate feedback control instructions and send them to the device, and record all historical data at the same time, supporting online learning and continuous optimization.

Images