Multi-modal artificial intelligence quality defect prediction method
Through the multimodal artificial intelligence quality defect prediction method, combined with real-time and historical data, feature extraction and prediction are carried out, and early warning mechanism is established, which solves the problem of insufficient single modal data in the existing technology, and achieves high efficiency for complex production environments. Accurate quality control.
Patent Information
- Application Number
- CN202411511062.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing quality defect prediction methods rely mostly on single modal data, and cannot effectively predict potential defects in complex production environments, and have shortcomings in real-time and accuracy, which cannot meet the quality control needs of modern production for efficient and high-precision.
The multimodal artificial intelligence quality defect prediction method is adopted to evaluate the data accuracy by obtaining real-time multimodal data (text, audio, image, sensor data) and historical data, perform feature extraction and model prediction, and establish an early warning mechanism. Dynamically adjust the warning parameters to achieve efficient and accurate defect prediction and early warning.
A comprehensive monitoring and analysis of quality defects is achieved, prediction errors are reduced, and a flexible early warning mechanism is established, which can quickly respond to potential defects, reduce false alarms and missed reports, and improve the understanding and control capabilities of the production process.
Smart Images

Figure CN119025870A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence manufacturing technology, and in particular to a multimodal artificial intelligence quality defect prediction method. Background Art
[0002] In recent years, with the rapid development of technologies such as artificial intelligence (AI), machine learning, and deep learning, data analysis and pattern recognition capabilities have been significantly improved. These technologies provide new solutions for the prediction and monitoring of quality defects. In particular, the application of multimodal data, by integrating text, audio, image, and sensor data, can obtain more comprehensive information from multiple dimensions and improve the understanding of products and production processes. For example, image data can be used to detect product appearance defects, audio data can be used to monitor the operating status of equipment, and sensor data can reflect changes in the production environment in real time.
[0003] However, despite the huge potential of multimodal data in quality inspection, current technical solutions still have some shortcomings. Many existing quality defect prediction methods often rely only on single-modal data and lack a comprehensive analysis of the data, resulting in insufficient prediction capabilities for potential defects in complex production environments. In addition, existing methods also have a large room for improvement in terms of real-time and accuracy, and cannot meet the modern production requirements for efficient and high-precision quality control. Therefore, there is an urgent need for a quality defect prediction method that integrates multimodal data, has efficient feature extraction and accurate prediction capabilities, in order to cope with the rapidly changing production environment and complex quality management challenges. Summary of the invention
[0004] The purpose of the present invention is to provide a multimodal artificial intelligence quality defect prediction method to solve the above-mentioned problems.
[0005] The purpose of the present invention can be achieved through the following technical solutions: The multimodal artificial intelligence quality defect prediction method includes the following steps: S1: acquiring real-time multimodal data, comparing the real-time multimodal data with historical multimodal data, evaluating the accuracy of the data, and dividing the multimodal data into accurate data and inaccurate data according to the evaluation results; Among them, multimodal data includes: text data, audio data, image data and sensor data; S2: Based on accurate data, extract features from multimodal data, use the extracted multimodal data features to establish a quality defect prediction model, and predict quality defects; S3: Based on the prediction results, calculate the deviation between the predicted value and the actual value, judge the accuracy of the prediction, and divide the predicted value into accurate prediction and inaccurate prediction; S4: Based on accurate prediction, an early warning mechanism is used to warn of multimodal data with potential defects.
[0006] As a further solution of the present invention: the acquiring of real-time multimodal data, comparing the real-time multimodal data with the historical multimodal data, and evaluating the accuracy of the data specifically include: Acquire real-time multimodal data from different sources, including text data, audio data, image data, and sensor data; Among them, sensor data includes data collected by any sensor; Extract corresponding features from real-time multimodal data and historical multimodal data, including image features, text features, audio features, and sensor data features; For numerical features, such as sensor data features, the Euclidean distance calculation expression is used to calculate the deviation between real-time sensor data and historical sensor data; Wherein, the calculation expression is: ; In the formula, Represents the statistical characteristics of real-time sensor data, represents the statistical characteristics of historical sensor data, represents the statistical characteristics of the i-th real-time sensor data, represents the statistical features of the i-th historical sensor data, i represents the number of statistical features of the sensor data, Indicates the data deviation between real-time sensor data and historical sensor data; For text features and image features, cosine similarity is used to calculate the deviation between real-time text features and historical text features, and cosine similarity is used to calculate the deviation between real-time image features and historical image features; Wherein, the expression for calculating the cosine similarity is: ; In the formula, Indicates the deviation between real-time text data and real-time image data and historical text data and historical image data, Represents real-time text data and real-time image data, Representing historical text data and historical image data; For time series data, such as audio data, a dynamic time warping algorithm is used to calculate the deviation between real-time audio data and historical audio data; The calculation expression of the dynamic time warping algorithm is: ; In the formula, represents the local distance of the series, represents the global minimum distance, that is, the deviation between the real-time audio data and the historical audio data, Indicates the minimum value.
[0007] As a further solution of the present invention: the dividing of the multimodal data into accurate data and inaccurate data according to the evaluation results specifically includes: Judging real-time multimodal data , and Whether it exceeds the corresponding preset threshold, if so, it is recorded as inaccurate data, if not, it is recorded as accurate data.
[0008] As a further solution of the present invention: based on accurate data, feature extraction is performed on multimodal data, and the extracted multimodal data features are used to establish a quality defect prediction model, which specifically includes: Standardize multimodal data. Each modality of data needs to be aligned in the time dimension. All the features of the extracted multimodal data are directly concatenated to form a unified feature vector, denoted as ; Unify the eigenvectors Input into a fully connected neural network; The input layer accepts a unified feature vector of multimodal fusion , whose dimensions are ; For each hidden layer, the input is processed through a linear transformation and a nonlinear activation function; Assuming that the neural network has l hidden layers, the output calculation expression of the lth layer is: ; In the formula, l represents the number of hidden layers, represents the weight matrix of the lth layer, represents the bias vector of the lth layer, represents the output of the l-1th layer, represents the activation function; For the output layer, which is used to predict the results of quality defects, the output can be a binary classification including the presence of defects and the absence of defects.
[0009] As a further solution of the present invention: the prediction of quality defects and calculation of the probability of existence of quality defects specifically include: For binary classification tasks, the Softmax function is used for classification and the probability of quality defects is calculated; The calculation expression for the probability of the existence of the calculated quality defect is: ; In the formula, represents the probability of defect existence, =1 indicates a defect. =0 means no defect, represents the weight matrix of the output layer, represents the bias of the output layer, represents the output of the last hidden layer.
[0010] As a further solution of the present invention: the calculation of the deviation between the predicted value and the actual value according to the prediction result specifically includes: Use the cross entropy loss function to calculate the error between the predicted value and the actual value; Among them, the calculation expression of the cross entropy loss function is: ; In the formula, represents the error value between the predicted value and the actual value, a represents the multimodal data for prediction in the ath group, and a is a positive integer greater than 0. Indicates that the multimodal data results predicted by group a are defective. =1 means that the data in group a is defective. =0 means that there is no defect in the data of group a.
[0011] As a further solution of the present invention: the determination of the accuracy of the prediction, dividing the prediction value into an accurate prediction and an inaccurate prediction specifically includes: Determine the error between the predicted value and the actual value Is it greater than or equal to the error value threshold between the corresponding preset predicted value and the actual value? If so, it is recorded as an inaccurate prediction; if not, it is recorded as an accurate prediction.
[0012] As a further solution of the present invention: the early warning mechanism based on accurate prediction for multimodal data with potential defects specifically includes: Establish a set of early warning rule base based on accurate prediction results and historical data; The early warning rule library is optimized and updated according to the characteristics of the multimodal data and the historical data of quality defect prediction; Set the warning level and dynamically adjust the thresholds of various warning parameters according to the fluctuation of real-time data and the trend of historical data; Through continuous monitoring of multimodal data, once the system detects that the current real-time data meets the warning conditions in the warning rule library, the warning is triggered immediately; The warning levels are: Low-level warning: It is only recorded in the system and prompts the operator to conduct preliminary inspection, but no immediate action is taken; Intermediate warning: notify relevant personnel to conduct further inspection and confirmation, and require the production line to be suspended or slowed down; Advanced warning: Shut down the machine immediately and notify the superior management and maintenance team, and initiate emergency response procedures; When the warning is confirmed to be correct, the system records the warning event and stores the data in the historical database to optimize future warning rules; When the warning is a false alarm, the system adjusts the corresponding warning rules or thresholds according to the actual situation to reduce the occurrence of false alarms; The system records each warning event, including time, warning content, warning level, and triggered multimodal data features.
[0013] Beneficial effects of the present invention: (1) By integrating multiple modal data such as text, audio, image and sensor, the method of the present invention realizes comprehensive monitoring and analysis of quality defects; different types of data sources complement each other to form a richer understanding of the production process and product quality; sensor data can reflect changes in the production environment and process parameters in real time, while image and audio data can provide an assessment of product appearance and sound characteristics; by using algorithms such as Euclidean distance, cosine similarity and dynamic time warping, the present invention can effectively identify the deviation between real-time data and historical data, thereby accurately judging the validity of the data; by standardizing multimodal data and extracting features, the quality defect prediction model constructed can significantly reduce prediction errors; (2) The method of the present invention establishes a complete early warning mechanism to deal with potential quality defect risks. By analyzing the accurate prediction results, the system can set the early warning level and formulate corresponding response strategies; the early warning mechanism is divided into multiple levels, and the early warnings from low to high levels correspond to different processing methods, which provides enterprises with flexible response capabilities; when the system recognizes that the real-time data meets the early warning conditions, it can quickly issue an alarm and record each early warning event. These data not only help to respond immediately, but also provide valuable historical data for subsequent analysis; the threshold of the early warning parameters is dynamically adjusted so that the system can maintain sensitivity in different production stages and conditions, ensuring a rapid response to potential problems; by comparing each early warning event with the actual situation, the system can continuously optimize the early warning rules, thereby reducing the occurrence of false alarms and missed alarms. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The present invention will be further described below in conjunction with the accompanying drawings.
[0015] Figure 1 It is a flowchart of the specific steps of the multimodal artificial intelligence quality defect prediction method of the present invention; Figure 2 It is a flowchart of establishing a quality defect prediction model in the present invention; Figure 3 It is a flowchart of the quality defect early warning processing in the present invention. DETAILED DESCRIPTION
[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0017] See also Figure 1 As shown, the present invention is a multimodal artificial intelligence quality defect prediction method, comprising the following steps: S1: acquiring real-time multimodal data, comparing the real-time multimodal data with historical multimodal data, evaluating the accuracy of the data, and dividing the multimodal data into accurate data and inaccurate data according to the evaluation results; Among them, multimodal data includes: text data, audio data, image data and sensor data; S2: Based on accurate data, extract features from multimodal data, use the extracted multimodal data features to establish a quality defect prediction model, and predict quality defects; S3: Based on the prediction results, calculate the deviation between the predicted value and the actual value, judge the accuracy of the prediction, and divide the predicted value into accurate prediction and inaccurate prediction; S4: Based on accurate prediction, an early warning mechanism is used to warn of multimodal data with potential defects.
[0018] In S1, real-time multimodal data is obtained, the real-time multimodal data is compared with historical multimodal data, the accuracy of the data is evaluated, and the multimodal data is divided into accurate data and inaccurate data according to the evaluation results. Specifically, the following are included: Acquire real-time multimodal data from different sources, including text data, audio data, image data, and sensor data; Among them, sensor data includes data collected by any sensor; Align real-time multimodal data and historical multimodal data according to timestamps to ensure that multimodal data in the same time window can be compared; Extract corresponding features from real-time multimodal data and historical multimodal data, including image features, text features, audio features, and sensor data features; For numerical features, such as sensor data features, the Euclidean distance calculation expression is used to calculate the deviation between real-time sensor data and historical sensor data; Wherein, the calculation expression is: ; In the formula, Represents the statistical characteristics of real-time sensor data, represents the statistical characteristics of historical sensor data, represents the statistical characteristics of the i-th real-time sensor data, represents the statistical features of the i-th historical sensor data, i represents the number of statistical features of the sensor data, i=1,2,...,n, Indicates the data deviation between real-time sensor data and historical sensor data; For text features and image features, cosine similarity is used to calculate the deviation between real-time text features and historical text features, and cosine similarity is used to calculate the deviation between real-time image features and historical image features; Wherein, the expression for calculating the cosine similarity is: ; In the formula, Indicates the deviation between real-time text data and real-time image data and historical text data and historical image data, Represents real-time text data and real-time image data, Representing historical text data and historical image data; For time series data, such as audio data, a dynamic time warping algorithm is used to calculate the deviation between real-time audio data and historical audio data; The calculation expression of the dynamic time warping algorithm is: ; In the formula, represents the local distance of the series, represents the global minimum distance, that is, the deviation between the data, Indicates the minimum value; Judging real-time multimodal data , and Whether it exceeds the corresponding preset threshold, if yes, it is recorded as inaccurate data, if no, it is recorded as accurate data; It should be noted that by judging the deviation between real-time multimodal data and historical multimodal data, it is possible to effectively judge whether the source of real-time data is accurate and whether the source of multimodal data is accurate. , and The size of the deviation can accurately identify the accurate data.
[0019] See also Figure 2As shown, in S2, based on accurate data, feature extraction is performed on multimodal data, and the extracted multimodal data features are used to establish a quality defect prediction model. The prediction of quality defects specifically includes: Standardize multimodal data. Each modality of data needs to be aligned in the time dimension. If the data of different modalities are collected at different times, use interpolation or time window methods to align them; All the features of the extracted multimodal data are directly concatenated to form a unified feature vector, denoted as ; Unify the eigenvectors Input into a fully connected neural network; The input layer accepts a unified feature vector of multimodal fusion , whose dimensions are ; For each hidden layer, the input is processed through a linear transformation (weight matrix multiplication) and a nonlinear activation function (ReLU); Assuming that the neural network has l hidden layers, the output calculation expression of the lth layer is: ; In the formula, l represents the number of hidden layers, represents the weight matrix of the lth layer, represents the bias vector of the lth layer, represents the output of the l-1th layer, represents the activation function; For the output layer, it is used to predict the results of quality defects. The output can be a binary classification including the presence of defects and the absence of defects. For binary classification tasks, the Softmax function is used for classification and the probability of quality defects is calculated; The calculation expression of the probability of the existence of the calculation defect is: ; In the formula, represents the probability of defect existence, =1 indicates a defect. =0 means no defect, represents the weight matrix of the output layer, represents the bias of the output layer, represents the output of the last hidden layer; It should be noted that: represents the probability of defect existence, =1 indicates a defect. =0 means there is no defect. According to the output result of the output layer in the prediction model, it can be judged whether there is a quality defect, thus achieving accurate identification of the quality.
[0020] In S3, based on the prediction results, the deviation between the predicted value and the actual value is calculated to determine the accuracy of the prediction. The predicted value is divided into accurate prediction and inaccurate prediction. Specifically, it includes: Use the cross entropy loss function to calculate the error between the predicted value and the actual value; Among them, the calculation expression of the cross entropy loss function is: ; In the formula, Represents the error value between the predicted value and the actual value, a represents the multimodal data for prediction in the ath group, and a is a positive integer greater than 0. Indicates that the multimodal data results predicted by group a are defective. =1 means that the data in group a is defective. =0 means that there is no defect in the data of group a; The error between the predicted value and the actual value Compare with the error value threshold between the corresponding preset predicted value and the actual value; If the error between the predicted value and the actual value If the error value between the corresponding preset predicted value and the actual value is greater than or equal to the threshold, it means that the corresponding prediction result is inaccurate and is recorded as an inaccurate prediction; If the error between the predicted value and the actual value If the error value between the corresponding preset predicted value and the actual value is less than the threshold, it means that the corresponding prediction result is accurate and is recorded as an accurate prediction; It should be noted that: by calculating the error value between the predicted value and the actual value , accurately identify the accuracy of the prediction results, and when the error value between the predicted value and the actual value The smaller the value, the more accurate the corresponding model prediction result.
[0021] See also Figure 3 As shown, in S4, based on accurate prediction, the early warning mechanism is used to warn multimodal data with potential defects, specifically including: Establish a set of early warning rule base based on the existing accurate prediction results and historical data; The early warning rule library is optimized and updated according to the characteristics of multimodal data and historical data of defect prediction, covering all known factors that may cause quality defects; Setting warning levels; According to the fluctuation of real-time data and the trend of historical data, the thresholds of various warning parameters are adjusted dynamically. This can adapt to different production scenarios and avoid frequent false alarms or missed alarms. Taking the data from the temperature sensor as an example, if the historical data has large fluctuations within a specific time period, the early warning system can adjust the upper and lower temperature limits of that period to avoid unnecessary alarms; Through continuous monitoring of multimodal data, once the system detects that the current real-time data meets a warning condition in the warning rule library, an alert is triggered immediately; The warning levels are: Low-level warning: It is only recorded in the system and prompts the operator to conduct preliminary inspection, but no immediate action is taken; Intermediate warning: notify relevant personnel to conduct further inspection and confirmation, and require the production line to be suspended or slowed down; Advanced warning: Shut down the machine immediately and notify the superior management and maintenance team, and initiate emergency response procedures; When the warning is confirmed to be correct, the system records the warning event and stores the data in the historical database to optimize future warning rules; When the warning is a false alarm, the system adjusts the corresponding warning rules or thresholds according to the actual situation to reduce the occurrence of similar false alarms; The system records every warning event, including time, warning content, warning level, and multimodal data features of the trigger; Through long-term statistical analysis of these data, the early warning model of quality defects can be further optimized to improve the accuracy and response efficiency of the system; When a high-level warning is triggered, the system allows operators to intervene manually to manually confirm whether the warning needs to be processed or ignored. The confirmed data feedback will be further used for model optimization.
[0022] Working principle of the present invention: Through effective comparison and analysis of real-time and historical data, accurate prediction of product quality defects is achieved. The method first obtains real-time multimodal data, including text data, audio data, image data and sensor data, and compares it with historical multimodal data to evaluate the accuracy of the data, and then divides the data into accurate data and inaccurate data. In order to ensure the effectiveness of the model, feature extraction is performed only based on accurate data. This process involves standardizing data of different modes to ensure alignment in the time dimension, then extracting the feature vectors of each mode, splicing them into a unified feature vector, and then inputting a fully connected neural network for quality defect prediction. The probability of the existence of quality defects is calculated by using the Softmax function to judge the accuracy of the prediction result. Based on the error value between the predicted value and the actual value, the cross entropy loss function is used for evaluation, so that the prediction results are divided into accurate predictions and inaccurate predictions. Further, for the accurate prediction results, an early warning mechanism is established to identify potential defects. The early warning mechanism includes an optimized and updated early warning rule base, and the early warning parameter threshold is dynamically adjusted according to the multimodal data characteristics and historical data to ensure the effectiveness of real-time monitoring. When the system detects that real-time data meets the warning conditions, it triggers a warning in time. The warning is divided into three levels to ensure a quick response to potential defects. During this process, the system also records each warning event to optimize future warning rules. The overall method combines the characteristics of multimodal data and uses advanced machine learning technology to improve the accuracy and timeliness of quality defect prediction, which has significant practical value and promotion prospects.
[0023] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.
[0024] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.
[0025] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.
[0026] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0027] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.
Claims
1. A multimodal artificial intelligence quality defect prediction method, characterized in that: The following steps are involved: S1: acquiring real-time multimodal data, comparing the real-time multimodal data with historical multimodal data, evaluating the accuracy of the data, and dividing the multimodal data into accurate data and inaccurate data according to the evaluation results; Among them, multimodal data includes: text data, audio data, image data and sensor data; S2: Based on accurate data, extract features from multimodal data, use the extracted multimodal data features to establish a quality defect prediction model, and predict quality defects; S3: Based on the prediction results, calculate the deviation between the predicted value and the actual value, judge the accuracy of the prediction, and divide the predicted value into accurate prediction and inaccurate prediction; S4: Based on accurate prediction, an early warning mechanism is used to warn of multimodal data with potential defects.
2. The multimodal artificial intelligence quality defect prediction method according to claim 1 is characterized in that: The obtaining of real-time multimodal data, comparing the real-time multimodal data with historical multimodal data, and evaluating the accuracy of the data specifically include: Acquire real-time multimodal data from different sources, including text data, audio data, image data, and sensor data; Among them, sensor data includes data collected by any sensor; Extract corresponding features from real-time multimodal data and historical multimodal data, including image features, text features, audio features, and sensor data features; For numerical features, such as sensor data features, the Euclidean distance calculation expression is used to calculate the deviation between real-time sensor data and historical sensor data; Wherein, the calculation expression is: ; In the formula, Represents the statistical characteristics of real-time sensor data, represents the statistical characteristics of historical sensor data, represents the statistical characteristics of the i-th real-time sensor data, represents the statistical features of the i-th historical sensor data, i represents the number of statistical features of the sensor data, Indicates the data deviation between real-time sensor data and historical sensor data; For text features and image features, cosine similarity is used to calculate the deviation between real-time text features and historical text features, and cosine similarity is used to calculate the deviation between real-time image features and historical image features; Wherein, the expression for calculating the cosine similarity is: ; In the formula, Indicates the deviation between real-time text data and real-time image data and historical text data and historical image data, Represents real-time text data and real-time image data, Representing historical text data and historical image data; For time series data, such as audio data, a dynamic time warping algorithm is used to calculate the deviation between real-time audio data and historical audio data; The calculation expression of the dynamic time warping algorithm is: ; In the formula, represents the local distance of the series, represents the global minimum distance, that is, the deviation between the real-time audio data and the historical audio data, Indicates the minimum value.
3. The multimodal artificial intelligence quality defect prediction method according to claim 1 is characterized in that: The dividing of the multimodal data into accurate data and inaccurate data according to the evaluation results specifically includes: Judging real-time multimodal data , and Whether it exceeds the corresponding preset threshold, if so, it is recorded as inaccurate data, if not, it is recorded as accurate data.
4. The multimodal artificial intelligence quality defect prediction method according to claim 1 is characterized in that: The method of extracting features from multimodal data based on accurate data and establishing a quality defect prediction model using the extracted multimodal data features specifically includes: Standardize multimodal data. Each modality of data needs to be aligned in the time dimension. All the features of the extracted multimodal data are directly concatenated to form a unified feature vector, denoted as ; Unify the eigenvectors Input into a fully connected neural network; The input layer accepts a unified feature vector of multimodal fusion , whose dimensions are ; For each hidden layer, the input is processed through a linear transformation and a nonlinear activation function; Assuming that the neural network has l hidden layers, the output calculation expression of the lth layer is: ; In the formula, l represents the number of hidden layers, represents the weight matrix of the lth layer, represents the bias vector of the lth layer, represents the output of the l-1th layer, represents the activation function; For the output layer, which is used to predict the results of quality defects, the output can be a binary classification including the presence of defects and the absence of defects.
5. The multimodal artificial intelligence quality defect prediction method according to claim 1, characterized in that: The prediction of quality defects and calculation of the probability of existence of quality defects specifically include: For binary classification tasks, the Softmax function is used for classification and the probability of quality defects is calculated; The calculation expression for the probability of the existence of the calculated quality defect is: ; In the formula, represents the probability of defect existence, =1 indicates a defect. =0 means no defect, represents the weight matrix of the output layer, represents the bias of the output layer, represents the output of the last hidden layer.
6. The multimodal artificial intelligence quality defect prediction method according to claim 1 is characterized in that: The step of calculating the deviation between the predicted value and the actual value according to the prediction result specifically includes: Use the cross entropy loss function to calculate the error between the predicted value and the actual value; Among them, the calculation expression of the cross entropy loss function is: ; In the formula, Represents the error value between the predicted value and the actual value, a represents the multimodal data for prediction in the ath group, and a is a positive integer greater than 0. Indicates that the multimodal data results predicted by group a are defective. =1 means that the data in group a is defective. =0 means that there is no defect in the data of group a.
7. The multimodal artificial intelligence quality defect prediction method according to claim 1 is characterized in that: The determination of the accuracy of the prediction and the classification of the prediction values into accurate predictions and inaccurate predictions specifically include: Determine the error between the predicted value and the actual value Is it greater than or equal to the error value threshold between the corresponding preset predicted value and the actual value? If so, it is recorded as an inaccurate prediction; if not, it is recorded as an accurate prediction.
8. The multimodal artificial intelligence quality defect prediction method according to claim 1 is characterized in that: The method of providing early warning for multimodal data with potential defects based on accurate prediction through an early warning mechanism specifically includes: Establish a set of early warning rule base based on accurate prediction results and historical data; The early warning rule library is optimized and updated according to the characteristics of the multimodal data and the historical data of quality defect prediction; Set warning levels and dynamically adjust the thresholds of various warning parameters based on real-time data fluctuations and historical data change trends; Through continuous monitoring of multimodal data, once the system detects that the current real-time data meets the warning conditions in the warning rule library, the warning is triggered immediately; The warning levels are: Low-level warning: It is only recorded in the system and prompts the operator to conduct preliminary inspection, but no immediate action is taken; Intermediate warning: Notify relevant personnel to conduct further inspection and confirmation, and require the production line to be suspended or slowed down; Advanced warning: Shut down the machine immediately and notify the superior management and maintenance team, and initiate emergency response procedures; When the warning is confirmed to be correct, the system records the warning event and stores the data in the historical database to optimize future warning rules; When the warning is a false alarm, the system adjusts the corresponding warning rules or thresholds according to the actual situation to reduce the occurrence of false alarms; The system records each warning event, including time, warning content, warning level, and triggered multimodal data features.
Citation Information
Patent Citations
Multi-modal deep learning-based power generation equipment abnormity integrated identification method and equipment
CN114463594A
Method for identifying abnormal working conditions of electric smelting furnace for magnesia based on multi-modal modulation algorithm
CN115033724A
Fault prediction method and device based on multi-modal data, equipment and storage medium
CN116184988A
Island photovoltaic operation and maintenance fault reason analysis method and system based on multi-modal data
CN117520924A
Risk prediction method based on multi-modal data fusion
CN117708746A
Cited By
Numerical data compression and storage method and device and storage medium
CN119377228A
Intelligent monitoring method for on-orbit observation quality of meteorological satellite remote sensing instrument
CN120259903A
Truck steel structure quality prediction method and system based on welding result
CN120562157A
A truck steel structure quality prediction method and system based on welding results
CN120562157B
Consistency comparison method and system for multi-mode electronic signed files
CN120599630A
