Bundle belt detection method and system based on multi-modal data analysis

Abstract

The invention relates to a bunching belt detection method and system based on multi-modal data analysis, and the method comprises the steps: collecting a multi-modal signal of a connection point of an automobile bunching belt through a sensor group, and obtaining a multi-modal signal of the automobile bunching belt; preprocessing the multi-modal signal to obtain preprocessed multi-modal data; fusing the multi-modal data preprocessed by using a multi-modal data fusion algorithm to form a comprehensive feature vector; analyzing the comprehensive feature vector by using a machine learning algorithm, identifying the health state of the automobile wire harness connection point, and determining the fault type according to the health state; based on the fault type, predicting the fault occurrence time of the connection point, and generating a fault occurrence probability and fault early warning information; and according to the fault occurrence probability and the fault early warning information, generating a real-time health state report, and uploading the multi-modal signal and the real-time health state report to a cloud platform. The method has the effect of improving the detection precision of the connection reliability of the bunching belt.

Description

Technical Field

[0001] The present application relates to the technical field of automobile manufacturing and maintenance, and particularly relates to a harness band detection method and system based on multi-modal data analysis. Background Art

[0002] As a core component of a vehicle's electrical system, the connection of automobile harness bands is directly related to the safety and stability of the vehicle. The quality of the harness band connection will affect the normal operation of the automobile electrical system and may even cause serious safety problems such as short circuits and electrical failures. However, existing harness band connection reliability detection methods mostly rely on traditional electrical detection or visual detection technologies, and these methods often have problems such as low detection accuracy and serious environmental noise interference. Traditional electrical detection methods can only detect when a connection point fails and cannot provide comprehensive and real-time monitoring data; while visual detection is limited by detection conditions and cannot accurately identify subtle changes in the connection point. The above-mentioned existing technical solutions have the following defects: Existing technologies generally face problems such as low accuracy, susceptibility to environmental interference, and inability to predict potential failures in the detection of harness band connection reliability, making it difficult to ensure the consistency of product quality, so there is room for improvement. Summary of the Invention

[0003] In order to improve the detection accuracy of the harness band connection reliability, the present application provides a harness band detection method and system based on multi-modal data analysis.

[0004] The first invention object of the present application is achieved through the following technical solutions: A harness band detection method based on multi-modal data analysis, the harness band detection method based on multi-modal data analysis includes: Collect multi-modal signals of the connection points of the automobile harness band through a sensor group to obtain the multi-modal signals of the automobile harness band; Preprocess the multi-modal signals to obtain preprocessed multi-modal data; Use a multi-modal data fusion algorithm to fuse the preprocessed multi-modal data to form a comprehensive feature vector; Use a machine learning algorithm to analyze the comprehensive feature vector, identify the health status of the connection points of the automobile harness band, and determine the fault type according to the health status; Based on the fault type, predict the fault occurrence time of the connection point, and generate a fault occurrence probability and a fault warning message; Generate a real-time health status report according to the fault occurrence probability and the fault warning message, and upload the multi-modal signals and the real-time health status report to a cloud platform.

[0005] By adopting the above technical solution, by collecting different types of signals for comprehensive monitoring, the working status of the automobile cable tie connection points can be observed from multiple angles, providing rich diagnostic data; pre-processing the multimodal signal through signal preprocessing technology helps to remove noise and interference in the original signal and improve signal quality; through the multimodal data fusion algorithm, different types of sensor data are weighted and fused to form a more accurate and comprehensive feature vector, which helps to improve the perception of the health status of the connection point and improve the accuracy of fault identification; analyzing the comprehensive feature vector through the machine learning algorithm helps to accurately identify the health status of the connection point and classify the fault according to the status, and can identify various potential faults and infer the specific fault type based on the characteristic pattern, thereby improving the accuracy and response speed of fault diagnosis; predicting the time and probability of the occurrence of the fault through the prediction model, judging the time and probability of the occurrence of the fault in advance, thereby realizing the early warning mechanism, reducing the downtime and maintenance costs caused by the occurrence of the fault, and improving the reliability and maintainability of the system; by generating a real-time health status report and uploading the data to the cloud platform, it helps to realize real-time remote monitoring of the automobile cable tie connection points.

[0006] In a preferred example, the present application may be further configured as follows: the multimodal signal of the automobile cable tie connection point is collected by the sensor group, and the multimodal signal of the automobile cable tie is obtained, which includes: A three-axis accelerometer is used to collect a vibration signal of a connection point of the automobile cable tie, wherein the sampling frequency of the three-axis accelerometer is not less than 10 kHz; The external environment temperature of the connection point of the automobile cable tie is collected in real time by a temperature sensor to generate a temperature signal, and the collection range of the temperature sensor is -40°C to 150°C; The humidity sensor collects the humidity change of the connection point of the automobile cable tie in real time to generate a humidity signal, and the collection range of the humidity sensor is 0% to 100%; The current sensor is used to monitor the current change of the connection point of the automobile cable tie in real time to generate a current signal, and the sampling accuracy of the current signal is 0.1A; The vibration signal, the temperature signal, the humidity signal and the current signal are statistically classified to obtain a multimodal signal of the automotive cable tie.

[0007] By adopting the above technical solution, the vibration signal of the automobile cable tie connection point is collected by a three-axis accelerometer. The vibration signal with a high-frequency sampling rate can accurately capture the tiny vibration changes of the automobile cable tie connection point, provide high-precision dynamic data, and ensure that the possible mechanical failures at the connection point can be detected and analyzed in time; the external ambient temperature of the automobile cable tie connection point is collected in real time by a temperature sensor, and the temperature monitoring with a wide temperature range can work stably in extreme temperature environments, and the ambient temperature changes of the automobile cable tie connection point are monitored in real time, providing basic data for analyzing poor contact or failures caused by temperature changes; the humidity changes of the automobile cable tie connection point are collected in real time by a humidity sensor, and the full-range humidity monitoring can detect Measure environmental conditions from dry to high humidity to help determine the effects of humidity changes on corrosion, oxidation or other faults that may be caused by automobile cable tie connection points, and improve the ability to warn of faults caused by environmental factors; use current sensors to monitor current changes at the automobile cable tie connection points in real time. High-precision current monitoring can accurately capture current fluctuations at the connection points and detect abnormal current changes in a timely manner, which helps to determine electrical faults and enhance the stability and safety of the electrical system; statistically classify signals, and multimodal data fusion combines vibration, temperature, humidity and current signals to achieve comprehensive diagnosis and fault warning of automobile cable tie connection points, enhance the system's comprehensive perception capabilities, and make fault prediction and fault type determination more accurate and comprehensive.

[0008] In a preferred example, the present application may be further configured as follows: the preprocessing of the multimodal signal to obtain the preprocessed multimodal data includes: Using a bandpass filter to remove noise in the multimodal signal, and performing smoothing processing on the current signal and the temperature signal to obtain a preliminarily processed multimodal signal; Through the wavelet transform formula: Extracting the time-frequency characteristics of the vibration signal to obtain characteristic data of the vibration signal, wherein W(f)(a,b) is the result of wavelet transform, f(t) is the signal to be analyzed, ψ(t) is the wavelet basis function, a is the scale parameter, b is the translation parameter, and t is time; The preliminarily processed multimodal signal and the characteristic data of the vibration signal are integrated to obtain the preprocessed multimodal data.

[0009] By adopting the above technical solution, noise is removed by a bandpass filter, and the current signal and temperature signal are smoothed, which can improve the quality of the signal and remove irrelevant noise components, making the subsequent feature extraction more accurate and obtaining a more accurate preliminary processed signal; the time-frequency characteristics of the vibration signal are extracted by wavelet transform, which effectively captures the local frequency change characteristics and instantaneous characteristics of the signal, and provides rich time-frequency feature data for subsequent feature analysis and pattern recognition; after the time-frequency feature data of the vibration signal are integrated with the multimodal signal, comprehensive data containing information such as time series, frequency and physical quantity changes can be obtained, providing richer signal input for subsequent feature analysis and fault diagnosis models.

[0010] In a preferred example, the present application may be further configured as follows: the multimodal data fusion algorithm is used to fuse the pre-processed multimodal data to form a comprehensive feature vector, including: Performing principal component analysis on the preprocessed multimodal data, extracting principal components from the preprocessed multimodal data, and obtaining analysis data; Calculating the contribution of each data in the analysis data under various failure modes, and assigning a weight to each data according to the contribution; According to the weighted average formula: Perform weighted summation to form the comprehensive feature vector, where F is the comprehensive feature vector, x i is the ith signal, w i is the weight of the ith signal, and N is the number of signals.

[0011] By adopting the above technical solution, through principal component analysis, the most representative characteristic components can be extracted from the preprocessed multimodal data, redundant components can be removed, the data structure can be simplified, and the efficiency of subsequent feature extraction and analysis can be effectively improved; by calculating the contribution of each feature and assigning corresponding weights, the influence of important features on fault mode recognition is effectively enhanced, thereby improving the accuracy and robustness of the fault diagnosis system; by generating a comprehensive feature vector through the weighted average method, the influence of different signals and features is effectively balanced, ensuring that key information is fully reflected, while reducing the interference of low-contribution features and improving the performance of subsequent models.

[0012] In a preferred example, the present application may be further configured as follows: the use of a machine learning algorithm to analyze the comprehensive feature vector, identify the health status of the automotive cable tie connection point, and determine the fault type according to the health status includes: Using a convolutional neural network to identify the pattern of vibration features in the comprehensive feature vector to generate a vibration pattern; Performing a joint analysis on the current feature and the temperature and humidity feature in the comprehensive feature vector to obtain an environmental fault state, wherein the health state includes the vibration mode and the environmental fault state; Based on the health status, the fault type is determined by analyzing in combination with the isolation forest algorithm.

[0013] By adopting the above technical solution, the vibration features in the comprehensive feature vector are identified through a convolutional neural network, and the patterns in the vibration signal are effectively extracted. It is possible to quickly and accurately identify whether the equipment is in a faulty state, thereby enhancing the accuracy and reliability of vibration fault identification. By jointly analyzing the current characteristics and the temperature and humidity characteristics, the health status of the equipment under different environmental conditions can be accurately evaluated, which further helps to determine whether the equipment has an environmental fault and improves the comprehensiveness and accuracy of fault diagnosis. By combining the isolation forest algorithm for health status analysis, the fault type of the equipment can be accurately identified, effectively improving the efficiency of fault diagnosis, and is particularly suitable for high-dimensional, multi-modal fault data analysis, thereby enhancing the robustness and interpretability of the diagnostic system.

[0014] In a preferred example, the present application may be further configured as follows: the prediction of the fault occurrence time of the connection point based on the fault type and the generation of the fault occurrence probability and fault warning information include: A prediction model is constructed using a hybrid network architecture combining a convolutional neural network and LSTM, and an adaptive reinforcement learning method is used to adjust the parameters of the prediction model according to real-time feedback during the prediction process; Based on the historical fault data, the multimodal signal and the health status, predict the state of the connection point to generate the fault occurrence time and fault occurrence probability of the connection point; The fault warning information is generated based on the fault occurrence time, the fault occurrence probability and the fault type.

[0015] By adopting the above technical solution, by combining convolutional neural networks and long short-term memory networks, and using adaptive reinforcement learning methods to adjust model parameters, the prediction model's ability to analyze multimodal signals is effectively improved, the model's adaptability and real-time response capabilities are enhanced, and thus the prediction accuracy and stability are improved; by combining historical fault data with multimodal signals, the state of the connection point can be accurately predicted, and the time and probability of fault occurrence can be generated, which is helpful for predictive maintenance, improves the accuracy of fault warning, and ensures stable operation of the system; by combining the time of fault occurrence, the probability of fault occurrence, and the type of fault, accurate fault warning information is generated, which can sound an alarm in advance, help equipment maintenance personnel take preventive measures in advance, effectively reduce equipment damage, and improve system reliability and work efficiency.

[0016] In a preferred example, the present application may be further configured as follows: generating the fault warning information based on the fault occurrence time, the fault occurrence probability and the fault type includes: Analyze the fault type, combine the fault occurrence time and the fault occurrence probability, classify the faults according to the preset impact priority, and generate a fault priority ranking; According to the fault priority ranking, the corresponding fault warning information is generated.

[0017] By adopting the above technical solution, by analyzing the fault type and combining the fault occurrence time and probability, and classifying and prioritizing the faults according to the preset impact priority, the system can give priority to the most urgent and serious faults, improve the efficiency and accuracy of fault handling, and avoid potential large-scale faults; through fault priority sorting, early warning information for different fault levels can be generated, so that fault handling personnel can quickly identify the most urgent faults, take timely prevention and handling measures, avoid equipment downtime or production losses, and improve fault response efficiency.

[0018] The second object of the invention is achieved by the following technical solutions: A cable tie detection system based on multimodal data analysis, the cable tie detection system based on multimodal data analysis comprising: A signal acquisition module, used for collecting multimodal signals of the connection points of the automobile cable ties through a sensor group to obtain the multimodal signals of the automobile cable ties; A data preprocessing module, used to preprocess the multimodal signal to obtain preprocessed multimodal data; A data fusion module, used to fuse the preprocessed multimodal data using the multimodal data fusion algorithm to form a comprehensive feature vector; An analysis module, configured to analyze the comprehensive feature vector using a machine learning algorithm, identify a health state of the automotive cable tie connection point, and determine a fault type based on the health state; A prediction module, used to predict the fault occurrence time of the connection point based on the fault type, and generate fault occurrence probability and fault warning information; The reporting module is used to generate a real-time health status report according to the fault occurrence probability and the fault warning information, and upload the multimodal signal and the real-time health status report to a cloud platform.

[0019] By adopting the above technical solution, by collecting different types of signals for comprehensive monitoring, the working status of the automobile cable tie connection points can be observed from multiple angles, providing rich diagnostic data; pre-processing the multimodal signal through signal preprocessing technology helps to remove noise and interference in the original signal and improve signal quality; through the multimodal data fusion algorithm, different types of sensor data are weighted and fused to form a more accurate and comprehensive feature vector, which helps to improve the perception of the health status of the connection point and improve the accuracy of fault identification; analyzing the comprehensive feature vector through the machine learning algorithm helps to accurately identify the health status of the connection point and classify the fault according to the status, and can identify various potential faults and infer the specific fault type based on the characteristic pattern, thereby improving the accuracy and response speed of fault diagnosis; predicting the time and probability of the occurrence of the fault through the prediction model, judging the time and probability of the occurrence of the fault in advance, thereby realizing the early warning mechanism, reducing the downtime and maintenance costs caused by the occurrence of the fault, and improving the reliability and maintainability of the system; by generating a real-time health status report and uploading the data to the cloud platform, it helps to realize real-time remote monitoring of the automobile cable tie connection points.

[0020] In summary, the present application includes at least one of the following beneficial technical effects: 1. By collecting different types of signals for comprehensive monitoring, the working status of the automotive cable tie connection points can be observed from multiple angles, providing rich diagnostic data; pre-processing multimodal signals through signal preprocessing technology helps to remove noise and interference in the original signal and improve signal quality; through the multimodal data fusion algorithm, different types of sensor data are weighted and fused to form a more accurate and comprehensive feature vector, which helps to improve the perception of the health status of the connection point and improve the accuracy of fault identification; 2. Analyzing the comprehensive feature vector through machine learning algorithms helps to accurately identify the health status of the connection points and classify faults according to the status. It can identify various potential faults and infer the specific fault type based on the characteristic pattern, thereby improving the accuracy and response speed of fault diagnosis; predicting the time and probability of fault occurrence through predictive models, judging the time and probability of fault occurrence in advance, thereby realizing an early warning mechanism, reducing downtime and maintenance costs caused by faults, and improving the reliability and maintainability of the system; generating real-time health status reports and uploading data to the cloud platform helps to achieve real-time remote monitoring of the automotive cable tie connection points. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a cable tie detection method based on multimodal data analysis in one embodiment of the present application; Figure 2is a flowchart for implementing step S10 in a cable tie detection method based on multimodal data analysis in one embodiment of the present application; Figure 3 is a flowchart for implementing step S20 in a cable tie detection method based on multimodal data analysis in one embodiment of the present application; Figure 4 is a flowchart for implementing step S30 in a cable tie detection method based on multimodal data analysis in one embodiment of the present application; Figure 5 is a flowchart for implementing step S40 in a cable tie detection method based on multimodal data analysis in one embodiment of the present application; Figure 6 is a flowchart for implementing step S50 in a cable tie detection method based on multimodal data analysis in one embodiment of the present application; Figure 7 is a flowchart for implementing step S53 in a cable tie detection method based on multimodal data analysis in one embodiment of the present application; Figure 8 It is a principle block diagram of a cable tie detection system based on multimodal data analysis in one embodiment of the present application; DETAILED DESCRIPTION

[0022] The present application is further described in detail below in conjunction with the accompanying drawings.

[0023] In one embodiment, if Figure 1 As shown, the present application discloses a cable tie detection method based on multimodal data analysis, which specifically includes the following steps: S10: Collect multimodal signals of the connection points of the automobile cable ties through the sensor group to obtain multimodal signals of the automobile cable ties.

[0024] Specifically, a group of high-precision sensors are arranged at different positions of the automotive cable ties. These sensors include temperature sensors, vibration sensors, pressure sensors, etc. Each sensor is responsible for collecting different types of signals. For example, the temperature sensor monitors the temperature changes of the connection point in real time, the vibration sensor captures the vibration signal of the connection point, and the pressure sensor measures the contact pressure and other physical parameters of the cable tie connection point. Through the coordinated work of these sensors, multiple signals can be obtained at the same time to reflect the working status of the connection point. The multiple signals obtained are integrated to obtain the multimodal signal of the automotive cable tie.

[0025] S20: Preprocess the multimodal signal to obtain preprocessed multimodal data.

[0026] Specifically, the original signal obtained from each sensor is first denoised using techniques such as low-pass filtering or median filtering to remove noise signals caused by environmental interference or sensor errors. Then, different signals are normalized to convert signals of different dimensions into a standardized format so that the data range of each signal is unified, which is convenient for subsequent fusion processing. In addition, the sensor data can be time-synchronized to ensure that the timestamps of different signals are aligned, which is convenient for subsequent data fusion analysis.

[0027] S30: Fusing the multimodal data preprocessed by the multimodal data fusion algorithm to form a comprehensive feature vector.

[0028] Specifically, by selecting a suitable multimodal data fusion algorithm, such as weighted averaging, principal component analysis or convolutional neural network, signals such as temperature, vibration and pressure are integrated into a comprehensive feature vector. First, a weight is assigned to each signal according to the correlation between different signals. The weight can be determined based on the analysis of signal characteristics and historical data. Then, feature extraction is performed on each signal, such as extracting key parameters such as temperature change rate, vibration spectrum characteristics, and pressure fluctuation amplitude. Finally, these features are calculated according to the weighted fusion algorithm to obtain a unified comprehensive feature vector, which can comprehensively reflect the health status of the connection point. S40: Use machine learning algorithms to analyze the comprehensive feature vectors to identify the health status of the vehicle cable tie connection points and determine the fault type based on the health status.

[0029] Specifically, first, use the labeled historical data set to select an appropriate machine learning model, such as a support vector machine, a random forest or a neural network model, to train the comprehensive feature vector. The model training process includes feature selection, division of training sets and test sets, model optimization and other steps. After the training is completed, the new comprehensive feature vector is input into the trained model. The classification results output by the model are used to determine whether a fault occurs at the connection point, and the type of fault is determined based on the output of the model, such as poor contact, breakage or other types of faults, and the causes of the faults are further analyzed.

[0030] S50: Based on the fault type, predict the fault occurrence time of the connection point, and generate the fault occurrence probability and fault warning information.

[0031] Specifically, first, historical fault data and machine learning models are used, combined with information on fault types, to predict the specific time when a fault will occur, using methods such as regression analysis or survival analysis. Based on the relationship model established between the fault type and historical data, the probability of failure at the connection point is calculated. Through the prediction results, the probability of occurrence of different fault types is obtained, and then detailed fault warning information is generated, including the possible time of failure and the severity of the failure, to provide a basis for subsequent maintenance decisions.

[0032] S60: Generate a real-time health status report based on the probability of failure occurrence and the fault warning information, and upload the multi-modal signals and the real-time health status report to the cloud platform.

[0033] Specifically, after the multi-modal signals of the connection points of the automotive wire harness tape are collected in real time, combining the predicted probability of failure occurrence and the warning information, by writing a health status evaluation algorithm, comprehensively analyzing the current signal characteristics and the fault warning information, automatically generate a real-time health status report. The report includes the current health status of the connection point, the fault type, the predicted time of failure occurrence, and the relevant recommended measures. After the report is generated, the multi-modal signals and the health status report are uploaded to the cloud platform through the wireless communication module. The cloud platform further processes and conducts real-time monitoring and analysis to provide decision-making support for the operation personnel.

[0034] By adopting the above technical solutions, through collecting different types of signals for comprehensive monitoring, it is possible to observe the working state of the connection points of the automotive wire harness tape from multiple angles and provide rich diagnostic data; through the signal preprocessing technology to preprocess the multi-modal signals, it helps to remove the noise and interference in the original signals and improve the signal quality; through the multi-modal data fusion algorithm, the sensor data of different types are weighted and fused to form a more accurate and comprehensive feature vector, which helps to improve the perception ability of the health status of the connection points and enhance the accuracy of fault identification; through the machine learning algorithm to analyze the comprehensive feature vector, it helps to accurately identify the health status of the connection points and classify the faults according to the status, and can identify various potential faults and infer the specific fault types according to the feature patterns, improving the accuracy and response speed of fault diagnosis; through the prediction model to predict the time and probability of failure occurrence, judge the time and probability of failure occurrence in advance, so as to realize the warning mechanism, reduce the downtime and maintenance costs caused by the occurrence of faults, and improve the reliability and maintainability of the system; by generating a real-time health status report and uploading the data to the cloud platform, it helps to realize the real-time remote monitoring of the connection points of the automotive wire harness tape.

[0035] In one embodiment, as Figure 2 shown, in step S10, that is, collect the multi-modal signals of the connection points of the automotive wire harness tape through the sensor group to obtain the multi-modal signals of the automotive wire harness tape, specifically including: S11: Use a three-axis accelerometer to collect the vibration signals of the connection points of the automotive wire harness tape, and the sampling frequency of the three-axis accelerometer is not less than 10 kHz.

[0036] Specifically, a high-precision triaxial accelerometer is installed at the connection point of the automotive wire harness strap. This accelerometer can measure acceleration signals simultaneously in the X, Y, and Z directions, and the sampling frequency is set to not less than 10 kHz to ensure that subtle vibration changes can be captured. Especially in high-frequency vibration and rapidly changing working environments, it can provide sufficiently fine vibration data. For example, during vehicle operation, the connection point of the wire harness strap may undergo small displacements due to vibration. The triaxial accelerometer can track these dynamic changes in real time through high-frequency sampling, obtain accurate vibration signals, and provide data support for subsequent health status assessment. S12: The external environmental temperature of the connection point of the automotive wire harness strap is collected in real time through a temperature sensor to generate a temperature signal. The collection range of the temperature sensor is from -40°C to 150°C.

[0037] Specifically, a suitable temperature sensor, such as a thermocouple or an RTD sensor, is selected and installed near the connection point of the automotive wire harness strap for real-time monitoring of the external environmental temperature of the connection point. The collection range of the selected temperature sensor is from -40°C to 150°C, which can cover the extreme temperature conditions that may be encountered in the vehicle's working environment. The sensor collects the environmental temperature signal in real time, converts it into an electrical signal output, and transmits it to the data processing end to ensure real-time monitoring of the environmental temperature, effectively detecting changes in the material properties of the connection point caused by temperature changes, and then judging possible faults or abnormalities. S13: The humidity change of the connection point of the automotive wire harness strap is collected in real time through a humidity sensor to generate a humidity signal. The collection range of the humidity sensor is from 0% to 100%.

[0038] Specifically, a humidity sensor is installed near the connection point of the automotive wire harness strap, and a suitable sensor type, such as a capacitive or resistive humidity sensor, is selected for real-time monitoring of the humidity change around the connection point. The collection range of the sensor covers the humidity range from 0% to 100%, which can cope with the environmental changes of the vehicle under different climatic conditions. The humidity signal collected by the sensor reflects the humidity level of the environment around the connection point, and humidity changes may affect the insulation performance of the connection point or cause faults such as short circuits in the circuit.

[0039] S14: The current change of the connection point of the automotive wire harness strap is monitored in real time through a current sensor to generate a current signal. The sampling accuracy of the current signal is 0.1 A.

[0040] Specifically, a high-precision current sensor, such as a Hall effect current sensor, is used to monitor the current changes at the connection points of the automotive cable ties in real time. The accuracy of the current sensor is set to 0.1A to ensure that tiny current fluctuations can be captured, especially current signal fluctuations generated by load changes, poor contact or faults at the connection points. The sensor collects current signals in real time and generates current data for subsequent analysis. Changes in current signals can often reflect whether a fault has occurred at the connection point, such as poor contact or a disconnected line.

[0041] S15: Statistically classify the vibration signal, the temperature signal, the humidity signal, and the current signal to obtain a multi-modal signal of the automobile cable tie.

[0042] Specifically, by comprehensively analyzing and processing the collected vibration signals, temperature signals, humidity signals and current signals, each signal is first statistically analyzed, including calculating the signal's mean, standard deviation, maximum value, minimum value and other statistical characteristic values. These characteristic values ​​can effectively characterize the change law and abnormal pattern of each signal. Then, these signal features are classified according to certain rules, and appropriate classification algorithms, such as K-means clustering or support vector machines, are used to group these signals by category to form multimodal signals.

[0043] In one embodiment, if Figure 3 As shown, in step S20, the multimodal signal is preprocessed to obtain preprocessed multimodal data, which specifically includes: S21: using a bandpass filter to remove noise from the multimodal signal, and performing smoothing on the current signal and the temperature signal to obtain a preliminarily processed multimodal signal.

[0044] Specifically, by designing a suitable bandpass filter, the noise of the collected multimodal signal is removed. First, according to the frequency characteristics of the multimodal signal, the passband range of the bandpass filter is set. For example, the passband range of the vibration signal is set to 10 Hz to 500 Hz to effectively remove the low-frequency baseline drift and high-frequency noise. Then, a digital filtering algorithm (such as an FIR or IIR filter) is used to process the signal point by point to filter out the noise components whose frequency components exceed the set range. On this basis, the current signal and the temperature signal are smoothed by a sliding average or exponential smoothing method. For example, a sliding average method with a sliding window size of 5 is used to calculate the average value of the data in the window point by point to eliminate transient fluctuations that may exist in the current signal and the temperature signal, thereby obtaining a smooth and noise-free preliminary processed multimodal signal.

[0045] S22: Through the wavelet transform formula: The time-frequency characteristics of the vibration signal are extracted to obtain the characteristic data of the vibration signal, where W(f)(a,b) is the result of wavelet transform, f(t) is the signal to be analyzed, ψ(t) is the wavelet basis function, a is the scale parameter, b is the translation parameter, and t is the time.

[0046] Specifically, the wavelet transform is used to extract the time-frequency features of the vibration signal. First, a suitable wavelet basis function is selected, such as Morlet wavelet or Haar wavelet. According to the time-varying characteristics of the vibration signal and the analysis requirements, the scale parameter a and translation parameter b of the wavelet transform are set. The scale parameter is adjusted in the frequency domain to analyze the characteristics of different frequency bands, and the local characteristics are located by the translation parameter in the time domain. Then, according to the wavelet transform formula The vibration signal f(t) is convolved point by point with the wavelet basis function ψ(t) to obtain the time-frequency distribution data, and the energy characteristics and transient characteristics within a specific frequency range are extracted. For example, the modulus of the wavelet coefficient can be calculated to identify the intensity of the vibration signal at a specific frequency, and the time-frequency characteristic parameters such as the main frequency and signal energy concentration can be obtained. These characteristic data reflect the dynamic behavior of the vibration signal.

[0047] S23: Integrate the preliminarily processed multimodal signal and the characteristic data of the vibration signal to obtain preprocessed multimodal data.

[0048] Specifically, the preliminary processed multimodal signal obtained by bandpass filtering and smoothing is integrated with the vibration signal feature data extracted by wavelet transform. First, the feature data of the preliminary processed multimodal signal and the vibration signal are normalized. For example, the maximum and minimum normalization method is used to map the data range to the [0,1] interval to ensure that different types of signals are on the same scale for easy comparison. Then, the preliminary processed signal and feature data are aligned according to the timestamp, and the misaligned data is padded by the interpolation method to ensure the time consistency of data integration. Subsequently, a data integration algorithm, such as a feature splicing method, is used to combine the feature values ​​of different signals into a comprehensive feature vector in the form of column splicing.

[0049] In one embodiment, if Figure 4 As shown, in step S30, the multimodal data preprocessed by the multimodal data fusion algorithm is fused to form a comprehensive feature vector, which specifically includes: S31: performing principal component analysis on the preprocessed multimodal data, extracting principal components from the preprocessed multimodal data, and obtaining analysis data.

[0050] Specifically, first, the preprocessed multimodal data is constructed into a data matrix, in which each column represents the eigenvector of a signal and each row corresponds to the data value at a sampling moment. Then, principal component analysis is performed to analyze the linear correlation between the signal features by calculating the covariance matrix of the data. Next, the eigenvalues ​​and eigenvectors of the covariance matrix are used to decompose the principal components with larger eigenvalues. These principal components can usually explain most of the variance of the original data. In this way, the most representative signal features are extracted, and finally an analysis data containing principal components is obtained. This data expresses the main features of the original multimodal data in fewer dimensions, while removing redundant and noise information.

[0051] S32: Calculate the contribution of each data in the analysis data under each failure mode, and assign a weight to each data according to the contribution.

[0052] Specifically, a fault pattern recognition algorithm is used to evaluate the performance of the analysis data under different fault modes. For example, machine learning models such as decision trees and support vector machines are used to input the analysis data under different fault modes into the model for training, so as to learn the importance or contribution of different data in each fault mode. Specifically, the contribution of each signal to fault pattern recognition can be quantified by calculating the influence of each signal feature, that is, by evaluating the feature importance in the model or using methods such as information gain. In this way, a weight can be assigned to each signal feature, and the weight reflects the contribution of the signal in a specific fault mode. Finally, a weight vector is obtained to quantify the importance of each data feature in different fault modes.

[0053] S33: According to the weighted average formula: Perform weighted summation to form a comprehensive feature vector, where F is the comprehensive feature vector, x i is the ith signal, w i is the weight of the ith signal, and N is the number of signals.

[0054] Specifically, the weighted average method is used to perform weighted summation on each signal feature according to the previously assigned weights. First, the value x of each signal feature is i The corresponding weight w i Multiply them to get the weighted value of each signal in the overall comprehensive feature vector, and then sum the weighted values ​​of all signals to get the final comprehensive feature vector F, where F is the comprehensive feature vector, representing the weighted sum of all signal features, x i is the value of the i-th signal, w i is the weight of the ith signal, N is the number of signals, and the comprehensive feature vector obtained by weighted summation can fully reflect the importance of each signal feature and the relevance of the fault mode.

[0055] In one embodiment, if Figure 5 As shown, in step S40, the comprehensive feature vector is analyzed using a machine learning algorithm to identify the health status of the automobile cable tie connection point, and the fault type is determined based on the health status, specifically including: S41: using a convolutional neural network to identify the pattern of vibration features in the comprehensive feature vector and generate a vibration pattern.

[0056] Specifically, before identifying the pattern, the historical vibration features are first extracted as the input data of the convolutional neural network. By designing appropriate convolutional layers, pooling layers and fully connected layers, a multi-level neural network structure is constructed. The convolutional layer is used to extract local features in the vibration signal, and the pooling layer is used to reduce the feature dimension and calculation amount while retaining the most important information. The convolutional neural network can automatically learn effective feature patterns by continuously optimizing parameters on the training data, and can identify the patterns of periodic changes and frequency characteristics in the vibration signal. Through the trained CNN model, the vibration features in the comprehensive feature vector are pattern recognized and the vibration pattern is output. The vibration pattern can reflect the type and characteristics of the vibration signal and help analyze whether the vibration state is abnormal. For example, when the vibration mode is high-frequency and violent vibration, it may indicate that the cable tie connection point is loose or has poor contact.

[0057] S42: jointly analyze the current characteristics and temperature and humidity characteristics in the comprehensive characteristic vector to obtain the environmental fault state. The health state includes the vibration mode and the environmental fault state.

[0058] Specifically, the current features, temperature features and humidity features in the comprehensive feature vector are jointly analyzed. First, the features of the current signal are extracted. For example, abnormal current fluctuations can be identified by calculating the mean and variance of the current. At the same time, the temperature and humidity signals are analyzed. By calculating the changing trends of temperature and humidity and the correlation with the current features, it is determined whether there are potential fault hazards caused by environmental factors. Then, these features are correlated and analyzed. For example, multivariate statistical analysis methods such as principal component analysis or canonical correlation analysis are used to extract environmental fault states that can comprehensively reflect the changes in current, temperature and humidity. Finally, the vibration mode is combined to form an overall health status assessment.

[0059] S43: Based on the health status, combined with the isolation forest algorithm, analysis is performed to determine the fault type.

[0060] Specifically, based on the health status information that has been obtained, it is used as the input data of the isolation forest algorithm. The isolation forest separates the data by continuously randomly selecting features and building a tree structure, and determines whether it is abnormal based on the degree of isolation of the data. Abnormal data is a potential fault mode. Combined with historical fault data and known fault types, the isolation forest algorithm can analyze the current health status and determine the fault type. For example, if the health status is manifested as excessive temperature, large current fluctuations, and abnormal vibration, the isolation forest may identify it as an electrical fault type, and the linkage analysis of vibration patterns and environmental data further verifies the type of the fault. Finally, according to the fault type output by the algorithm.

[0061] In one embodiment, if Figure 6 As shown, in step S50, based on the fault type, the fault occurrence time of the connection point is predicted, and the fault occurrence probability and fault warning information are generated, specifically including: S51: A hybrid network architecture combining convolutional neural network and LSTM is used to build a prediction model, and an adaptive reinforcement learning method is used to adjust the parameters of the prediction model according to real-time feedback during the prediction process.

[0062] Specifically, we first design a hybrid network architecture that combines convolutional neural networks and long short-term memory networks to build a prediction model. In the prediction process, in order to improve the accuracy of the model, we use adaptive reinforcement learning, such as Q-learning or PPO algorithm, to adjust network parameters according to real-time feedback. For example, we dynamically adjust the learning rate or network structure through feedback information to ensure that the model can self-adjust and optimize as the environment changes. S52: Based on the historical fault data, multimodal signals and health status, the state of the connection point is predicted to generate the fault occurrence time and fault occurrence probability of the connection point.

[0063] Specifically, the model is trained using historical failure data, which includes the time when different connection points failed in the past, environmental factors, and health status data. These data constitute a training set for training the prediction model. By learning the patterns of these historical data, the failure time and failure probability of the connection point can be predicted from the current multimodal signal and health status. For example, when the vibration mode in the health state changes and the temperature and humidity signals fluctuate abnormally, combined with the historical failure data, multimodal signal and health status, the prediction model can determine that the connection point may fail in the next period of time, and predict the specific time point of the failure and the probability of the failure. The failure time can be obtained through regression analysis, and the failure probability is output by the model as a probability value, which indicates the possibility of failure within a certain period of time in the future.

[0064] S53: Generate fault warning information based on the fault occurrence time, fault occurrence probability and fault type.

[0065] Specifically, after obtaining the time and probability of a fault, the prediction results are converted into actual fault warning information. For example, when the probability of a fault exceeds a set threshold, the system will generate a warning message to indicate the time and type of possible fault. For example, when the model predicts that the probability of a fault at a certain connection point is higher than 90%, and the fault will occur within the next hour, a fault warning message is generated to inform maintenance personnel of the fault type and expected time of the connection point. In addition, by comparing the fault type with historical fault patterns, it can help determine the severity of the fault and decide whether to take temporary repairs or to shut down immediately for maintenance.

[0066] In one embodiment, if Figure 7 As shown, in step S53, fault warning information is generated based on the fault occurrence time, fault occurrence probability and fault type, specifically including: S531: Analyze the fault type, combine the fault occurrence time and the fault occurrence probability, classify the fault according to the preset impact priority, and generate a fault priority ranking.

[0067] Specifically, firstly, the historical data of different fault types are analyzed to determine the severity of each fault and its impact on the system. These impact priorities can be preset based on factors such as the potential harm of the fault type, the probability of occurrence, and the time urgency of the fault occurrence. For example, current overload and poor contact may correspond to high-priority and medium-priority faults, respectively, while excessive temperature or abnormal humidity may correspond to low-priority faults. Then, the occurrence time and probability of each fault type are combined, and the faults are classified and prioritized using a weighted algorithm to generate a fault priority ranking. For example, if the probability of a connection point failing within the next 10 minutes is 95%, and the fault type is current overload, its priority will be set to the highest and needs to be handled immediately. If the fault type is excessive humidity and the probability of occurrence is low, its priority will be relatively low.

[0068] S532: Generate corresponding fault warning information according to the fault priority ranking.

[0069] Specifically, after the fault priority ranking is generated, a series of corresponding fault warning information can be generated according to the preset priority level. First, each fault is arranged from high to low priority, and the timing and content of sending the warning information are determined according to the priority of the fault. For example, for the highest priority fault, the warning information is generated and sent immediately, and the content includes the time, type, possible impact and emergency handling suggestions of the fault. For lower priority faults, it can be set to delay the sending of warnings or only remind within a specific time period.

[0070] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0071] In one embodiment, a cable tie detection system based on multimodal data analysis is provided, and the cable tie detection system based on multimodal data analysis corresponds one-to-one to the cable tie detection method based on multimodal data analysis in the above embodiment. Figure 8 As shown, the cable tie detection system based on multimodal data analysis includes a signal acquisition module, a data preprocessing module, a data fusion module, an analysis module, a prediction module and a report module. The functional modules are described in detail as follows: the signal acquisition module is used to collect multimodal signals of the connection points of the automobile cable tie through a sensor group to obtain the multimodal signals of the automobile cable tie; A data preprocessing module, used to preprocess the multimodal signal to obtain preprocessed multimodal data; A data fusion module is used to fuse the multimodal data preprocessed by the multimodal data fusion algorithm to form a comprehensive feature vector; An analysis module, which is used to analyze the comprehensive feature vector using a machine learning algorithm to identify the health status of the automotive cable tie connection point and determine the fault type based on the health status; A prediction module is used to predict the fault occurrence time of the connection point based on the fault type and generate the fault occurrence probability and fault warning information; The reporting module is used to generate a real-time health status report based on the fault probability and fault warning information, and upload the multimodal signal and the real-time health status report to the cloud platform.

[0072] Optionally, the signal acquisition module includes: The vibration detection submodule is used to collect the vibration signal of the connection point of the automobile cable tie using a three-axis accelerometer, and the sampling frequency of the three-axis accelerometer is not less than 10kHz; The temperature detection submodule is used to collect the external ambient temperature of the connection point of the automobile cable tie in real time through a temperature sensor to generate a temperature signal. The temperature sensor has a collection range of -40°C to 150°C. The humidity detection submodule is used to collect the humidity changes of the connection points of the automobile cable ties in real time through the humidity sensor and generate a humidity signal. The collection range of the humidity sensor is 0% to 100%; The current detection submodule is used to monitor the current changes at the connection points of the automotive cable ties in real time through the current sensor and generate a current signal with a sampling accuracy of 0.1A. The statistical classification submodule is used to statistically classify the vibration signal, temperature signal, humidity signal and current signal to obtain the multimodal signal of the automobile cable tie.

[0073] Optionally, the data preprocessing module includes: A denoising submodule is used to remove noise from the multimodal signal by using a bandpass filter, and to smooth the current signal and the temperature signal to obtain a preliminarily processed multimodal signal; The wavelet transform submodule is used to transform the following wavelet transform formula: Extract the time-frequency characteristics of the vibration signal to obtain the characteristic data of the vibration signal, where W(f)(a,b) is the result of wavelet transform, f(t) is the signal to be analyzed, ψ(t) is the wavelet basis function, a is the scale parameter, b is the translation parameter, and t is the time; The data integration submodule is used to integrate the characteristic data of the preliminarily processed multimodal signal and the vibration signal to obtain preprocessed multimodal data.

[0074] Optionally, the data fusion module includes: The principal component analysis submodule is used to perform principal component analysis on the preprocessed multimodal data, extract the principal components in the preprocessed multimodal data, and obtain analysis data; The contribution calculation submodule is used to calculate the contribution of each data in the analysis data under various fault modes, and assign weights to each data according to the contribution; The weighted calculation submodule is used to calculate the weighted average formula: Perform weighted summation to form a comprehensive feature vector, where F is the comprehensive feature vector, x i is the ith signal, w i is the weight of the ith signal, and N is the number of signals.

[0075] Optional analysis modules include: A pattern recognition submodule, for identifying patterns of vibration features in a comprehensive feature vector using a convolutional neural network to generate a vibration pattern; The joint analysis submodule is used to jointly analyze the current characteristics and temperature and humidity characteristics in the comprehensive feature vector to obtain the environmental fault state. The health state includes the vibration mode and the environmental fault state. The type determination submodule is used to determine the fault type based on the health status and the isolation forest algorithm.

[0076] Optionally, the prediction module includes: The model building submodule is used to build a prediction model using a hybrid network architecture combining convolutional neural networks and LSTM, and to use adaptive reinforcement learning methods to adjust the parameters of the prediction model according to real-time feedback during the prediction process; The model training submodule is used to predict the state of the connection point based on historical fault data, multimodal signals and health status, and generate the fault occurrence time and probability of the connection point; The warning information generation submodule is used to generate fault warning information based on the fault occurrence time, fault occurrence probability and fault type.

[0077] Optionally, the submodule for generating early warning information includes: A priority determination unit is used to analyze the fault type, classify the faults according to the preset impact priority in combination with the fault occurrence time and the fault occurrence probability, and generate a fault priority ranking; Determine the corresponding warning information unit, which is used to generate corresponding fault warning information according to the fault priority sorting.

[0078] For the specific definition of a cable tie detection system based on multimodal data analysis, please refer to the definition of a cable tie detection method based on multimodal data analysis above, which will not be repeated here. Each module in the above-mentioned cable tie detection system based on multimodal data analysis can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0079] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

[0080] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A cable tie detection method based on multimodal data analysis, characterized in that: The cable tie detection method based on multimodal data analysis includes: Collecting multimodal signals of the connection points of the automobile cable ties through the sensor group to obtain the multimodal signals of the automobile cable ties; Preprocessing the multimodal signal to obtain preprocessed multimodal data; The preprocessed multimodal data is fused using the multimodal data fusion algorithm to form a comprehensive feature vector; Analyzing the comprehensive feature vector using a machine learning algorithm to identify a health state of the automotive cable tie connection point, and determining a fault type based on the health state; Based on the fault type, predict the fault occurrence time of the connection point, and generate the fault occurrence probability and fault warning information; A real-time health status report is generated according to the fault occurrence probability and the fault warning information, and the multimodal signal and the real-time health status report are uploaded to a cloud platform.

2. The cable tie detection method based on multimodal data analysis according to claim 1, characterized in that: The step of collecting the multimodal signal of the connection point of the automobile cable tie by the sensor group to obtain the multimodal signal of the automobile cable tie comprises: collecting the vibration signal of the connection point of the automobile cable tie by using a triaxial accelerometer, wherein the sampling frequency of the triaxial accelerometer is not less than 10 kHz; The external environment temperature of the connection point of the automobile cable tie is collected in real time by a temperature sensor to generate a temperature signal, and the collection range of the temperature sensor is -40°C to 150°C; The humidity sensor collects the humidity change of the connection point of the automobile cable tie in real time to generate a humidity signal, and the collection range of the humidity sensor is 0% to 100%; The current sensor is used to monitor the current change of the connection point of the automobile cable tie in real time to generate a current signal, and the sampling accuracy of the current signal is 0.1A; The vibration signal, the temperature signal, the humidity signal and the current signal are statistically classified to obtain a multimodal signal of the automotive cable tie.

3. The cable tie detection method based on multimodal data analysis according to claim 2, characterized in that: The preprocessing of the multimodal signal to obtain preprocessed multimodal data includes: Using a bandpass filter to remove noise in the multimodal signal, and performing smoothing processing on the current signal and the temperature signal to obtain a preliminarily processed multimodal signal; Through the wavelet transform formula: Extracting the time-frequency characteristics of the vibration signal to obtain characteristic data of the vibration signal, wherein W(f)(a,b) is the result of wavelet transform, f(t) is the signal to be analyzed, ψ(t) is the wavelet basis function, a is the scale parameter, b is the translation parameter, and t is time; The preliminarily processed multimodal signal and the characteristic data of the vibration signal are integrated to obtain the preprocessed multimodal data.

4. The cable tie detection method based on multimodal data analysis according to claim 1, characterized in that: The method of fusing the pre-processed multimodal data using a multimodal data fusion algorithm to form a comprehensive feature vector includes: Performing principal component analysis on the preprocessed multimodal data, extracting principal components from the preprocessed multimodal data, and obtaining analysis data; Calculating the contribution of each data in the analysis data under various failure modes, and assigning a weight to each data according to the contribution; According to the weighted average formula: Perform weighted summation to form the comprehensive feature vector, where F is the comprehensive feature vector, x i is the ith signal, w i is the weight of the ith signal, and N is the number of signals.

5. The cable tie detection method based on multimodal data analysis according to claim 1, characterized in that: The using of a machine learning algorithm to analyze the comprehensive feature vector, identifying the health status of the automotive cable tie connection point, and determining the fault type according to the health status includes: Using a convolutional neural network to identify the pattern of vibration features in the comprehensive feature vector to generate a vibration pattern; Performing a joint analysis on the current feature and the temperature and humidity feature in the comprehensive feature vector to obtain an environmental fault state, wherein the health state includes the vibration mode and the environmental fault state; Based on the health status, the fault type is determined by analyzing in combination with the isolation forest algorithm.

6. The cable tie detection method based on multimodal data analysis according to claim 1, characterized in that: The predicting of the fault occurrence time of the connection point based on the fault type and generating the fault occurrence probability and fault warning information includes: A prediction model is constructed using a hybrid network architecture combining a convolutional neural network and LSTM, and an adaptive reinforcement learning method is used to adjust the parameters of the prediction model according to real-time feedback during the prediction process; Based on the historical fault data, the multimodal signal and the health status, predict the state of the connection point to generate the fault occurrence time and fault occurrence probability of the connection point; The fault warning information is generated based on the fault occurrence time, the fault occurrence probability and the fault type.

7. The cable tie detection method based on multimodal data analysis according to claim 6, characterized in that: The generating the fault warning information based on the fault occurrence time, the fault occurrence probability and the fault type includes: Analyze the fault type, combine the fault occurrence time and the fault occurrence probability, classify the faults according to the preset impact priority, and generate a fault priority ranking; According to the fault priority ranking, the corresponding fault warning information is generated.

8. A cable tie detection system based on multimodal data analysis, characterized in that: The cable tie detection system based on multimodal data analysis includes: A signal acquisition module, used for collecting multimodal signals of the connection points of the automobile cable ties through a sensor group to obtain the multimodal signals of the automobile cable ties; A data preprocessing module, used to preprocess the multimodal signal to obtain preprocessed multimodal data; A data fusion module, used to fuse the preprocessed multimodal data using the multimodal data fusion algorithm to form a comprehensive feature vector; An analysis module, configured to analyze the comprehensive feature vector using a machine learning algorithm, identify a health state of the automotive cable tie connection point, and determine a fault type based on the health state; A prediction module, used to predict the fault occurrence time of the connection point based on the fault type, and generate fault occurrence probability and fault warning information; The reporting module is used to generate a real-time health status report according to the fault occurrence probability and the fault warning information, and upload the multimodal signal and the real-time health status report to a cloud platform.

Images