Automobile fastener state detection method and system based on artificial intelligence

Abstract

The invention discloses an automobile fastener state detection method and system based on artificial intelligence, and relates to the technical field of fastener state detection, the system comprises the following components: a machine vision module, an infrared thermal imaging module, an acoustic analysis module, a multi-modal data fusion processing module and a decision output module; fastener images are captured through the high-precision camera, appearance defect detection is conducted through the image processing and deep learning technology, the infrared thermal imaging module monitors the temperature abnormity of the fasteners, the overheating problem caused by loosening and overload is found and early warned in time, and the acoustic analysis module monitors the vibration sound of the fasteners, so that the detection accuracy of the fasteners is improved. The multi-modal data fusion processing module is used for judging the fastening state and abnormal noise of the fastener, the multi-modal data fusion processing module fuses data of all the modules, comprehensive evaluation of the state of the fastener is achieved through comprehensive feature extraction and a machine learning algorithm, and the decision output module outputs a state report and an early warning signal according to an evaluation result, automatically gives an alarm and provides processing suggestions.

Description

Technical Field

[0001] The present invention relates to the technical field of fastener detection, and in particular to an automobile fastener state detection method and system based on artificial intelligence. Background Art

[0002] With the rapid development of modern society, the status detection of automobile fasteners is very important. Fasteners, as common mechanical parts, play a vital role in automobile manufacturing. They are not only used to connect and fix key parts of the car, but also directly related to the structural safety and reliability of the car.

[0003] Existing traditional methods for detecting the status of automotive fasteners often rely on a single means. These methods have the problems of low detection accuracy, low efficiency and susceptibility to interference from human factors. When processing multi-source data, there is a lack of data fusion and processing technology, resulting in the failure to fully utilize the correlation and complementarity between data. They rely on manual judgment and experience accumulation and lack intelligent decision-making support. In complex situations, manual judgment is often difficult to guarantee accuracy and timeliness, increasing the risk of failure.

[0004] At present, traditional multimodal data fusion methods are usually static, using the same computational processing and fusion method for all multimodal inputs without considering the different computational requirements of different modal data. Therefore, it is very important to develop an artificial intelligence automotive fastener status detection method and system. Summary of the invention

[0005] The purpose of the present invention is to make up for the deficiencies of the prior art and to provide an automobile fastener status detection method and system based on artificial intelligence, which can realize comprehensive, real-time and accurate detection and evaluation of the automobile fastener status by integrating multiple advanced technologies.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: an artificial intelligence-based automobile fastener status detection method and system, the system comprising the following components: a machine vision module, an infrared thermal imaging module, an acoustic analysis module, a multimodal data fusion processing module and a decision output module;

[0007] The machine vision module is responsible for detecting the appearance defects of fasteners, capturing images of fasteners with a high-precision camera, identifying defects through image processing technology, and extracting and classifying features of images in combination with texture feature extraction methods;

[0008] The infrared thermal imaging module is used to monitor the temperature anomaly of the fasteners, obtain the temperature distribution diagram of the fastener surface through the infrared thermal imager, and analyze the temperature change law. When the fasteners are overheated due to looseness and overload, the infrared thermal imaging module can detect it in time and send out a warning signal;

[0009] The acoustic analysis module: by monitoring the vibration sound of the fastener, judge its fastening state and whether there is abnormal noise, use a high-sensitivity microphone to collect the vibration sound signal of the fastener, and extract the sound features through signal processing technology, and use the support vector machine SVM to classify the sound features to judge whether the fastening state of the fastener is abnormal;

[0010] The multimodal data fusion processing module is responsible for acquiring data from the three modules of machine vision, infrared thermal imaging and acoustic analysis, aligning the data from different sensors in time and space, using a multimodal fusion algorithm to perform fusion analysis on multi-source data, and classifying and making decisions on the fused features through a decision tree algorithm to obtain the status assessment results of the fasteners;

[0011] The decision output module outputs a fastener status assessment report and a warning signal according to the result of the multimodal data fusion processing module. When an abnormality is detected in the fastener, the system automatically sends an alarm signal and prompts corresponding treatment measures.

[0012] Furthermore, the machine vision module uses a high-precision camera to obtain images of fasteners in different environments, performs grayscale processing on the collected original images, converts the color images into grayscale images, and uses a median filter algorithm. The median filter calculation formula is: Output = Median {Pixel l i},i∈[1,N], where Pixel i Represents the i-th pixel value in the window, N is the total number of pixels in the window, and the gray-level co-occurrence matrix method is used as the feature description method for fastener surface texture analysis. The gray-level co-occurrence matrix P(i, j, d, θ) is calculated, where i and j represent gray values, d represents the distance between pixels, and θ represents the direction. The energy eigenvalue and contrast eigenvalue are calculated from the gray-level co-occurrence matrix. The energy eigenvalue calculation formula is: Energy = ∑ i ∑ j P(i,j) 2 , Contrast characteristic value calculation formula: Contras=∑ i ∑ j (ij) 2 P(i,j), through the change analysis of texture features, corrosion and pit defects are identified, and a dataset containing various defective and non-defective fastener images is constructed.

[0013] Furthermore, the infrared thermal imaging module selects an infrared thermal imager, sets the measurement range, resolution and frame rate parameters of the thermal imager according to the monitoring environment and the characteristics of the fastener, and performs a calibration operation to ensure the measurement accuracy of the thermal imager. The calibration formula is: corrected =T measured+ΔT, where T corrected is the calibrated temperature, T m easured is the measured temperature, ΔT is the calibration deviation, aim the infrared thermal imager at the fastener to ensure that the area to be monitored can be fully covered, start the thermal imager for image acquisition, obtain the real-time temperature distribution image of the fastener surface, process and analyze the obtained temperature distribution image, extract the temperature value of each part of the fastener, and calculate the average temperature parameter. The average value formula is: in, is the average temperature, n is the number of measurement points, T i is the temperature of the ith measurement point. Compare the temperature distribution images collected at different time points, analyze the temperature change trend over time, and use linear regression and other methods to fit the temperature change curve. The formula is: T=at+b, where T is temperature, t is time, a and b are fitting parameters. According to the preset temperature threshold, determine whether the fastener is overheated. Combined with the temperature change law, determine whether the overheating is caused by looseness and overload. When it is determined that the fastener temperature is abnormal, the infrared thermal imaging module immediately sends out a warning signal. The warning signal can be in the form of sound and light alarm, text message and email.

[0014] Furthermore, the acoustic analysis module selects a high-sensitivity microphone suitable for monitoring the vibration sound of the fastener, and installs it at a suitable position close to the fastener, and then calibrates the microphone to ensure that it can accurately collect sound signals. corrected =S measured +ΔS, where S corrected is the calibrated sound signal strength, S measured is the measured sound signal strength, ΔS is the calibration deviation, the microphone is started, the vibration sound of the fastener is monitored, and the sound signal is collected in real time. The collected sound signal is filtered and noise reduction preprocessed to improve the signal quality. The filtering formula can be expressed as, Where Y(n) is the output signal after filtering, h(k) is the filter coefficient, X(nk) is the input signal, and the characteristic parameters of the sound, frequency, amplitude and energy are extracted by using time domain, frequency domain and time-frequency domain analysis methods to calculate the short-time energy of the sound signal, E(n) = ∑ m＝n-N+1 x 2 (m), where E(n) is the short-time energy of the nth frame, x(m) is the sound signal, and N is the frame length. The extracted sound features are input into the trained SVM classifier. The decision function of SVM is expressed as: Where x is the input sound feature vector, x i is the support vector, y i is the corresponding category label, α i is the Lagrange multiplier, K(x i,x) is the kernel function, b is the bias term, and the output result of the SVM classifier is used to determine whether the fastening state of the fastener is abnormal.

[0015] Furthermore, the multimodal data fusion processing module obtains high-definition image data of the fastener from the machine vision module, obtains temperature distribution data on the surface of the fastener from the infrared thermal imaging module, obtains sound signal data of the fastener when working from the acoustic analysis module, performs spatial registration on the machine vision image data and the infrared thermal imaging data, performs timestamp correction on all sensor data, extracts visual features from the preprocessed image data, extracts thermal features from the infrared thermal imaging data, extracts acoustic features from the acoustic data, uses a multimodal fusion algorithm to fuse the extracted visual features, thermal features and acoustic features, uses a decision tree algorithm to classify the fused comprehensive features, identifies different status categories of the fastener, evaluates the status of the fastener according to the classification results, and generates a corresponding status evaluation report or visualization output.

[0016] Furthermore, the multimodal data fusion processing module uses a multimodal fusion algorithm to fuse the extracted visual features, thermal features and acoustic features. From the preprocessed machine vision image data, a convolutional neural network (CNN) method is used to extract visual features, and then from the infrared thermal imaging data, the statistical features of temperature distribution are extracted, and from the acoustic data, the frequency, amplitude, and energy acoustic features of sound are extracted. Convolutional neural network (CNN): Convolution layer:

[0017]

[0018] Among them, O c (x, y) is the value of the output feature map at channel c and position (x, y), b c is the bias of channel c, pooling layer: For the maximum pooling operation, assuming that the input feature map is I and the pooling window size is W p ×H p , the step length is S p , then the output features of the pooling layer can be expressed as: Among them, R xy is the pooling window area corresponding to the position (x, y) on the input feature map. The fully connected layer assumes that the input vector is: x = [x1, x2, ..., x n ], the weight matrix is ​​W = [w ij ], the bias vector is b=[b1,b2,…,b m ], then the output vector of the fully connected layer y=[y1,y2,…,y m ] can be calculated as: Among them, f is the activation function, and then the visual features, thermal features and acoustic features are fused by weighted summation. The fusion formula can be expressed as F=w1V+w2T+w3A, where F is the fused feature, V is the visual feature vector, T is the thermal feature vector, A is the acoustic feature vector, and w1, w2, and w3 are the corresponding weights respectively.

[0019] Furthermore, the decision tree algorithm of the multimodal data fusion processing module classifies the fused comprehensive features, identifies the different status categories of the fasteners, sorts the fused comprehensive feature data, and marks the corresponding fastener actual status category for each data sample, calculates the information gain of each feature to select the optimal split feature and split point. Assume that the data set D contains K categories, C k Represents the number of samples belonging to the kth class, and the information entropy calculation formula is: in is the proportion of samples belonging to the kth class, split according to feature A, which has n possible values ​​{a1, a2, ..., a n}, then the information gain brought by the split based on feature A is: Among them, D i The feature A takes the value a i The corresponding sample subset is selected according to the optimal features and splitting points. The data set is split into multiple subsets. Step 2 is repeated for each subset until the stopping condition is met. For the new fused feature data, the constructed decision tree starts from the root node and branches down according to the judgment of the feature value until it reaches the leaf node to obtain the predicted fastener status category. In practical applications, the construction and use of decision trees may be adjusted and optimized according to the specific data distribution and problem requirements.

[0020] Furthermore, the decision output module receives the comprehensive evaluation results from the multimodal data fusion processing module. When analyzing the evaluation results, it also detects whether there is an abnormal state. According to the severity and urgency of the abnormal state, the level of the warning signal is determined, and the system automatically issues a warning signal. The form of the warning signal includes but is not limited to sound alarms, screen pop-ups and SMS notifications. The system provides corresponding processing measures and records all issued warning signals and processing measures.

[0021] On the other hand, an artificial intelligence-based automobile fastener status detection method is characterized in that the method specifically comprises the following steps:

[0022] S100, using machine vision module to detect fastener appearance defects: using high-precision camera to obtain images of fasteners, using image processing technology to process the images combined with texture feature extraction methods, extracting image features and classifying them to identify appearance defects;

[0023] S200, uses infrared thermal imaging module to monitor abnormal fastener temperature: Use infrared thermal imager to obtain temperature distribution map of fastener surface, analyze temperature variation pattern, and timely detect and issue warning signal when fastener overheats due to looseness and overload;

[0024] S300 relies on the acoustic analysis module to determine the fastening status and abnormal noise of fasteners: a high-sensitivity microphone is used to collect the vibration sound signal of the fastener, the sound features are extracted through signal processing technology, and the support vector machine (SVM) is used to classify the sound features to determine whether the fastening status is abnormal;

[0025] S400 uses a multimodal data fusion processing module to integrate and analyze data: it collects data from three modules: machine vision, infrared thermal imaging, and acoustic analysis. It uses a multimodal fusion algorithm to fuse multi-source data from different sensors in time, and uses a decision tree algorithm to classify and make decisions on fusion features to obtain fastener status assessment results.

[0026] S500, gives evaluation results and warnings through the decision output module: outputs a fastener status evaluation report based on the results of the multimodal data fusion processing module. When an abnormality is detected, the system automatically sends an alarm signal and prompts corresponding treatment measures.

[0027] Furthermore, the S400 fuses the extracted visual features, thermal features and acoustic features, and the fusion formula is: F = w1V + w2T + w3A, where F is the fused feature, V is the visual feature vector, T is the thermal feature vector, A is the acoustic feature vector, and w 1、 w 2、 w3 is the corresponding weight. The fused features are input into the decision tree algorithm to calculate indicators such as information gain and Gini index, select the optimal split features and split points, and build a decision tree. The information gain formula is: Where D is the data set, A is the feature, Ent is the entropy, Values(A) is all possible values ​​of feature A, and D v It is the data subset when feature A takes the value v.

[0028] Compared with the prior art, the automotive fastener status detection method and system based on artificial intelligence has the following beneficial effects:

[0029] 1. The automobile fastener status detection system of the present invention realizes comprehensive, real-time and accurate detection and evaluation of fastener status by integrating multiple advanced artificial intelligence technologies, which not only improves the efficiency and accuracy of automobile maintenance, but also reduces the safety hazards and maintenance costs caused by fastener failures, providing strong support for the intelligent development of the automobile industry.

[0030] 2. The automobile fastener status detection system of the present invention shows significant beneficial effects in detection accuracy, detection efficiency, early warning and automatic processing capabilities, decision support capabilities and adaptability. These advantages make the system have broad application prospects and important application value in the field of automobile fastener status detection.

[0031] Other advantages, objectives and features of the present invention will be set forth in part in the following description and, in part, will be apparent to those skilled in the art based on an examination of the following or may be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0033] Figure 1 This is a process operation diagram of an automobile fastener status detection system based on artificial intelligence;

[0034] Figure 2 The present invention is a process operation diagram of an automobile fastener status detection method based on artificial intelligence. DETAILED DESCRIPTION

[0035] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0036] Embodiment 1

[0037] First: select a high-precision camera as the core equipment of the machine vision module, which can clearly capture the image details of the fasteners. Select a high-performance infrared thermal imager as the infrared thermal imaging module, which can accurately monitor the temperature changes of the fasteners in different environments. Equip a high-sensitivity microphone to the acoustic analysis module and install it in a suitable position to accurately capture the vibration sound of the fasteners. Install the above modules at key workstations on the production line to ensure full coverage of the fastener production and inspection processes. Integrate the machine vision module, infrared thermal imaging module and acoustic analysis module with the production line control system to realize real-time data transmission and processing. Introduce a multimodal data fusion processing module and a decision output module to build a complete fastener status detection system.

[0038] Then: capture the image of the fastener in real time, perform grayscale and filter preprocessing operations to remove image noise and interference, and use the median filter algorithm. The median filter calculation formula is: Output = Median {Pixel i},i∈[1,N], where Pixel i Represents the i-th pixel value in the window, N is the total number of pixels in the window, and the texture feature extraction method is used to extract and classify the preprocessed image, identify the appearance defects of the fastener, obtain the temperature distribution map of the fastener surface, extract temperature information through image processing technology, calculate the average temperature, and analyze the temperature change law over time. The temperature change curve is fitted using linear regression and other methods. When the fastener temperature exceeds the preset threshold, it is judged to be overheated. The vibration sound signal of the fastener is collected, and filtering and noise reduction preprocessing operations are performed to improve the signal quality. The characteristic parameters of the sound signal are extracted, and the support vector machine (SVM) machine learning algorithm is used to classify the sound features to determine whether the fastening state of the fastener is abnormal. The decision function of SVM is expressed as: Where x is the input sound feature vector, x i is the support vector, y i is the corresponding category label, α i is the Lagrange multiplier, K(x i ,x) is the kernel function, b is the bias term, and the output result of the SVM classifier is used to determine whether the fastening state of the fastener is abnormal.

[0039] Next: Multimodal data fusion and decision data alignment and fusion. The multimodal data fusion processing module receives data from the three modules of machine vision, infrared thermal imaging and acoustic analysis, aligns them in time and space, and uses a multimodal fusion algorithm to fuse visual features, thermal features and acoustic features to form a comprehensive feature vector. Convolutional Neural Network (CNN): Convolutional layer:

[0040]

[0041] Among them, O c (x, y) is the value of the output feature map at channel c and position (x, y), b c is the bias of channel c. The decision tree algorithm is used to classify the fused comprehensive features and identify the different status categories of fasteners. Information entropy: It is used to measure the uncertainty of the data set. The formula is Where S is the sample set, c is the number of categories, p i is the proportion of samples of the i-th category in the set. Gini index: used to evaluate the impurity of samples in the data set. The formula is Based on the classification results, the status of the fasteners is evaluated and a corresponding status evaluation report is generated.

[0042] Finally: The decision output module automatically generates a fastener status assessment report based on the results of the multimodal data fusion processing module, including the location, status, abnormality type and severity information of the fastener. When an abnormality is detected in the fastener, the system automatically issues a warning signal. The signal form includes but is not limited to sound alarms, screen pop-ups and SMS notifications. The warning signal should clearly indicate the type, location and urgency of the abnormality so that the operator can respond quickly and take appropriate measures. The system responds according to the abnormality type and severity of the fastener.

[0043] Embodiment 2

[0044] First: Install high-precision cameras at each key location on the production line to clearly capture images of fasteners and transmit data to the image processing system. The system continuously captures images to ensure that no key information is missed. Set up an infrared thermal imager near the production line. According to the size, shape and distribution of the fasteners, adjust the measurement range, resolution and frame rate parameters of the thermal imager to ensure that the thermal imager can clearly display the temperature distribution map of the fasteners. The infrared thermal imager performs non-contact temperature monitoring on the fasteners and obtains the temperature distribution map of its surface in real time through infrared thermal imaging technology. Infrared thermal imaging technology is usually based on the law of thermal radiation, the Stefan-Boltzmann law: E = σT 4 , where E is the radiation emittance, σ is the Stefan-Boltzmann constant, and T is the absolute temperature of the object, which can accurately reflect the temperature change of the fastener.

[0045] Then: install high-sensitivity microphones around the production line. These microphones should be able to accurately capture the vibration sound of the fasteners. Perform filtering and noise reduction preprocessing operations on the collected sound signals to improve the signal quality. Y(f) = H(f)·X(f), where X(f) is the frequency domain representation of the input signal, Y(f) is the frequency domain representation of the filtered output signal, and H(f) is the frequency response of the filter. The characteristic parameters of the sound are extracted through time domain, frequency domain and time-frequency domain analysis methods. The sound features are classified using the trained SVM classifier to determine the tightening state of the fasteners and whether there is abnormal noise. The decision function can be expressed as: The SVM classifier is trained and optimized based on a large amount of sample data to ensure the accuracy and reliability of the judgment.

[0046] Next: align the data from the machine vision, infrared thermal imaging and acoustic analysis modules in time and space, extract visual features from the machine vision image data, extract thermal features from the infrared thermal imaging data, and extract acoustic features from the acoustic data. Use a multimodal fusion algorithm to perform fusion analysis on the extracted features, comprehensively evaluate the status of the fasteners, and use a decision tree algorithm to classify and make decisions on the fused features to obtain the fastener status assessment results.

[0047] Finally: Evaluation report: Based on the results of the multimodal data fusion processing module, a fastener status evaluation report is automatically generated. The report contains detailed information of the fastener, test results, evaluation conclusions and recommended treatment measures.

[0048] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. An automotive fastener status detection system based on artificial intelligence, characterized in that: The system includes a machine vision module, an infrared thermal imaging module, an acoustic analysis module, a multimodal data fusion processing module, and a decision output module: The machine vision module is responsible for detecting the appearance defects of fasteners, capturing images of fasteners with a high-precision camera, identifying defects through image processing technology, and extracting and classifying features of images in combination with texture feature extraction methods; The infrared thermal imaging module is used to monitor the temperature anomaly of the fasteners, obtain the temperature distribution diagram of the fastener surface through the infrared thermal imager, and analyze the temperature change law. When the fasteners are overheated due to looseness and overload, the infrared thermal imaging module can detect it in time and send out a warning signal; The acoustic analysis module: by monitoring the vibration sound of the fastener, judge its fastening state and whether there is abnormal noise, use a high-sensitivity microphone to collect the vibration sound signal of the fastener, and extract the sound features through signal processing technology, and use the support vector machine SVM to classify the sound features to judge whether the fastening state of the fastener is abnormal; The multimodal data fusion processing module is responsible for acquiring data from the three modules of machine vision, infrared thermal imaging and acoustic analysis, aligning the data from different sensors in time and space, using a multimodal fusion algorithm to perform fusion analysis on multi-source data, and classifying and making decisions on the fused features through a decision tree algorithm to obtain the status assessment results of the fasteners; The decision output module outputs a fastener status assessment report and a warning signal according to the result of the multimodal data fusion processing module. When an abnormality is detected in the fastener, the system automatically sends an alarm signal and prompts corresponding treatment measures.

2. The automotive fastener status detection system based on artificial intelligence according to claim 1 is characterized in that: The machine vision module uses a high-precision camera to obtain images of fasteners in different environments, grayscales the collected original images, converts color images into grayscale images, and uses a median filter algorithm. The median filter calculation formula is: Output = Median {Pixel l i },i∈[1,N], where Pixel i Represents the i-th pixel value in the window, N is the total number of pixels in the window, and the gray-level co-occurrence matrix method is used as the feature description method for fastener surface texture analysis. The gray-level co-occurrence matrix P(i, j, d, θ) is calculated, where i and j represent gray values, d represents the distance between pixels, and θ represents the direction. The energy eigenvalue and contrast eigenvalue are calculated from the gray-level co-occurrence matrix. The energy eigenvalue calculation formula is: Energy = ∑ i ∑ j P(i,j) 2 , Contrast characteristic value calculation formula: Contras=∑ i ∑ j (ij) 2 P(i,j), through the change analysis of texture features, corrosion and pit defects are identified, and a dataset containing various defective and non-defective fastener images is constructed.

3. The automotive fastener status detection system based on artificial intelligence according to claim 1 is characterized in that: The infrared thermal imaging module selects an infrared thermal imager, sets the measurement range, resolution and frame rate parameters of the thermal imager according to the monitoring environment and the characteristics of the fastener, and performs a calibration operation to ensure the measurement accuracy of the thermal imager. The calibration formula is: corrected =T measured +ΔT, where T corrected is the calibrated temperature, T m easured is the measured temperature, ΔT is the calibration deviation, aim the infrared thermal imager at the fastener to ensure that the area to be monitored can be fully covered, start the thermal imager for image acquisition, obtain the real-time temperature distribution image of the fastener surface, process and analyze the obtained temperature distribution image, extract the temperature value of each part of the fastener, and calculate the average temperature parameter. The average value formula is: in, is the average temperature, n is the number of measurement points, T i is the temperature of the ith measurement point. Compare the temperature distribution images collected at different time points, analyze the temperature change trend over time, and use linear regression and other methods to fit the temperature change curve. The formula is: T=at+b, where T is temperature, t is time, a and b are fitting parameters. According to the preset temperature threshold, determine whether the fastener is overheated. Combined with the temperature change law, determine whether the overheating is caused by looseness and overload. When it is determined that the fastener temperature is abnormal, the infrared thermal imaging module immediately sends out a warning signal. The warning signal can be in the form of sound and light alarm, text message and email.

4. The automotive fastener status detection system based on artificial intelligence according to claim 1 is characterized in that: The acoustic analysis module selects a high-sensitivity microphone suitable for monitoring the vibration sound of the fastener, and installs it in a suitable position close to the fastener, and then calibrates the microphone to ensure that it can accurately collect sound signals. corrected =S measured +ΔS, where S corrected is the calibrated sound signal strength, S measured is the measured sound signal strength, ΔS is the calibration deviation, the microphone is started, the vibration sound of the fastener is monitored, and the sound signal is collected in real time. The collected sound signal is filtered and noise reduction preprocessed to improve the signal quality. The filtering formula can be expressed as, Where Y(n) is the output signal after filtering, h(k) is the filter coefficient, X(nk) is the input signal, and the characteristic parameters of the sound, frequency, amplitude and energy are extracted by using time domain, frequency domain and time-frequency domain analysis methods to calculate the short-time energy of the sound signal, E(n) = ∑ m＝n-N+1 x 2 (m), where E(n) is the short-time energy of the nth frame, x(m) is the sound signal, and N is the frame length. The extracted sound features are input into the trained SVM classifier. The decision function of SVM is expressed as: Where x is the input sound feature vector, x i is the support vector, y i is the corresponding category label, α i is the Lagrange multiplier, K(x i ,x) is the kernel function, b is the bias term, and the output result of the SVM classifier is used to determine whether the fastening state of the fastener is abnormal.

5. The automotive fastener status detection system based on artificial intelligence according to claim 1 is characterized in that: The multimodal data fusion processing module obtains high-definition image data of the fastener from the machine vision module, obtains temperature distribution data on the surface of the fastener from the infrared thermal imaging module, obtains sound signal data of the fastener when working from the acoustic analysis module, performs spatial registration on the machine vision image data and the infrared thermal imaging data, performs timestamp correction on all sensor data, extracts visual features from the preprocessed image data, extracts thermal features from the infrared thermal imaging data, extracts acoustic features from the acoustic data, adopts a multimodal fusion algorithm to fuse the extracted visual features, thermal features and acoustic features, uses a decision tree algorithm to classify the fused comprehensive features, identifies different status categories of the fastener, evaluates the status of the fastener according to the classification results, and generates a corresponding status evaluation report or visual output.

6. The automotive fastener status detection system based on artificial intelligence according to claim 1 is characterized in that: The multimodal data fusion processing module adopts a multimodal fusion algorithm to fuse the extracted visual features, thermal features and acoustic features. From the preprocessed machine vision image data, a convolutional neural network (CNN) method is used to extract visual features, and then from the infrared thermal imaging data, the statistical features of temperature distribution are extracted, and from the acoustic data, the frequency, amplitude, and energy acoustic features of sound are extracted. Convolutional neural network (CNN): Convolution layer: Among them, O c (x, y) is the value of the output feature map at channel c and position (x, y), b c is the bias of channel c, pooling layer: For the maximum pooling operation, assuming that the input feature map is I and the pooling window size is W p ×H p , the step length is S p , then the output features of the pooling layer can be expressed as: x ′ ,y ′ ) Among them, R xy is the pooling window area corresponding to the position (x, y) on the input feature map. The fully connected layer assumes that the input vector is: x = [x1, x2, ..., x n ], the weight matrix is ​​W = [w ij ], the bias vector is b=[b1,b2,…,b m ], then the output vector of the fully connected layer y=[y1,y2,…,y m ] can be calculated as: Among them, f is the activation function, and then the visual features, thermal features and acoustic features are fused by weighted summation. The fusion formula can be expressed as F=w1V+w2T+w3A, where F is the fused feature, V is the visual feature vector, T is the thermal feature vector, A is the acoustic feature vector, and w1, w2, and w3 are the corresponding weights respectively.

7. The automotive fastener status detection system based on artificial intelligence according to claim 1 is characterized in that: The decision tree algorithm of the multimodal data fusion processing module classifies the fused comprehensive features and identifies the different status categories of fasteners. The fused comprehensive feature data is sorted and the corresponding actual status category of fasteners is marked for each data sample. The information gain of each feature is calculated to select the optimal split feature and split point. Assume that the data set D contains K categories, C k Represents the number of samples belonging to the kth class, and the information entropy calculation formula is: (p k ),in is the proportion of samples belonging to the kth class, split according to feature A, which has n possible values ​​{a1, a2, ..., a n }, then the information gain brought by the split based on feature A is: Among them, D i The feature A takes the value a i The corresponding sample subset is selected according to the optimal features and splitting points. The data set is split into multiple subsets. Step 2 is repeated for each subset until the stopping condition is met. For the new fused feature data, the constructed decision tree starts from the root node and branches down according to the judgment of the feature value until it reaches the leaf node to obtain the predicted fastener status category. In practical applications, the construction and use of decision trees may be adjusted and optimized according to the specific data distribution and problem requirements.

8. The automotive fastener status detection system based on artificial intelligence according to claim 1 is characterized in that: The decision output module receives the comprehensive evaluation results from the multimodal data fusion processing module. When analyzing the evaluation results, it also detects whether there is an abnormal state. According to the severity and urgency of the abnormal state, the level of the warning signal is determined. The system automatically issues a warning signal. The form of the warning signal includes but is not limited to sound alarms, screen pop-ups and SMS notifications. The system provides corresponding processing measures and records all issued warning signals and processing measures.

9. An automobile fastener status detection method based on artificial intelligence, characterized in that: The specific steps of this method are: S100, using machine vision module to detect fastener appearance defects: using high-precision camera to obtain images of fasteners, using image processing technology to process the images combined with texture feature extraction methods, extracting image features and classifying them to identify appearance defects; S200, uses infrared thermal imaging module to monitor abnormal fastener temperature: Use infrared thermal imager to obtain temperature distribution map of fastener surface, analyze temperature variation pattern, and timely detect and issue warning signal when fastener overheats due to looseness and overload; S300 relies on the acoustic analysis module to determine the fastening status and abnormal noise of fasteners: a high-sensitivity microphone is used to collect the vibration sound signal of the fastener, the sound features are extracted through signal processing technology, and the support vector machine (SVM) is used to classify the sound features to determine whether the fastening status is abnormal; S400 uses a multimodal data fusion processing module to integrate and analyze data: it collects data from three modules: machine vision, infrared thermal imaging, and acoustic analysis. It uses a multimodal fusion algorithm to fuse multi-source data from different sensors in time, and uses a decision tree algorithm to classify and make decisions on fusion features to obtain fastener status assessment results. S500, gives evaluation results and warnings through the decision output module: outputs a fastener status evaluation report based on the results of the multimodal data fusion processing module. When an abnormality is detected, the system automatically sends an alarm signal and prompts corresponding treatment measures.

10. The method for detecting automobile fastener status based on artificial intelligence according to claim 9, characterized in that: The S400 fuses the extracted visual features, thermal features and acoustic features, and the fusion formula is: F = w1V + w2T + w3A, where F is the fused feature, V is the visual feature vector, T is the thermal feature vector, A is the acoustic feature vector, and w 1、 w 2、 w3 is the corresponding weight. The fused features are input into the decision tree algorithm to calculate indicators such as information gain and Gini index, select the optimal split features and split points, and build a decision tree. The information gain formula is: Where D is the data set, A is the feature, Ent is the entropy, Values(A) is all possible values ​​of feature A, and D v It is the data subset when feature A takes the value v.

Images