A multi-modal fusion-based FPC connector quality detection method and system

Abstract

The application relates to the technical field of electronic equipment detection, and discloses a FPC connector quality detection method and system based on multi-modal fusion, which comprises the following steps: obtaining an original multi-modal data set containing electrical stress, thermal stress and mechanical stress, and performing filtering and time sequence alignment preprocessing to obtain a standardized multi-physical field sequence; constructing a three-dimensional space field strength distribution map according to the sequence, and identifying field strength anomalies by using density clustering to accurately locate potential defects; learning the time sequence evolution mode based on multi-time observation values of the defect position to predict the future diffusion trend of the defects; and comprehensively modeling and evaluating by fusing the diffusion trend, real-time field strength and environmental variables to generate a dynamically updated and final quality detection report. The application can realize root cause diagnosis of connector defects, accurate fault positioning in a three-dimensional space, and accurate prediction of future evolution trends, and significantly improves the detection accuracy and predictability.

Description

Technical Field

[0001] This invention relates to the field of electronic device condition monitoring technology, and in particular to a method and system for quality inspection of FPC connectors based on multimodal fusion. Background Technology

[0002] In the fields of electronic manufacturing and system integration, connector quality inspection is a core component to ensure the long-term stable operation and signal integrity of equipment. With increasingly stringent reliability requirements from industries such as communications, automotive electronics, and aerospace, developing high-precision, highly predictive connector defect detection technology has become a crucial technological support for ensuring the development of high-end manufacturing.

[0003] In existing technologies, connector quality inspection often relies on measurement methods based on a single physical dimension. For example, it may determine the circuit status solely by analyzing electrical parameters such as time-domain reflectometry (TDR), or by using imaging techniques like X-rays to obtain static two-dimensional structural information. While these methods can identify existing obvious defects to some extent, they have fundamental limitations. On one hand, they separate the intrinsically related physical fields such as electrical, thermal, and mechanical stress, failing to capture the true causes and dynamic evolution of defects under multi-field coupling. At a deeper level, existing technologies lack real-time insight into the dynamic changes in the internal field strength distribution of connectors and fail to establish a correlation mechanism between the physical morphology of defects (such as the three-dimensional structure of microcracks) and their evolutionary trends under multi-physics interaction (such as the propagation rate and direction of cracks). This results in an inability to fundamentally and accurately assess and predict the potential risks of defects.

[0004] In summary, existing connector quality inspection methods are unable to integrate multi-physics information to diagnose the root causes of defects and predict their evolution trends, resulting in technical problems such as insufficient detection accuracy and lack of predictability. Summary of the Invention

[0005] This invention provides a method and system for quality inspection of FPC connectors based on multimodal fusion, so as to improve the accuracy of defect diagnosis and the reliability of evolution trend prediction in complex multiphysics environments.

[0006] Firstly, to address the aforementioned technical problems, this invention provides a method for quality inspection of FPC connectors based on multimodal fusion, comprising:

[0007] Obtain the raw multimodal dataset inside the connector, which includes electrical stress, thermal stress, and mechanical stress data;

[0008] The original multimodal dataset is subjected to mean filtering and temporal alignment to obtain a standardized multiphysics sequence;

[0009] spatial feature extraction and temporalization construction are performed on the standardized multi-physical field sequence to obtain a spatial field intensity distribution map;

[0010] If the field intensity distribution in the spatial field intensity distribution map exceeds a preset field intensity threshold, density clustering and spatial correlation analysis are performed to obtain a potential defect area list;

[0011] The spatial data of the potential defect area list is obtained, a connector internal three-dimensional structure model is constructed, structure analysis and abnormal point positioning are performed, and accurate defect position coordinates are obtained.

[0012] Multi-time observation values of the accurate defect position coordinates are obtained, time series evolution mode learning and future trend prediction are performed on the observation values, and a future evolution path of the defect is obtained.

[0013] The gradient change of the spatial field intensity distribution map is calculated, and comprehensive modeling is performed in combination with the future evolution path to obtain a dynamic field intensity update sequence.

[0014] According to the dynamic field intensity update sequence, in combination with the real-time obtained field intensity distribution data, diffusion rate calculation and quality assessment are performed to obtain a final quality detection report.

[0015] Preferably, the mean filtering and time series alignment of the original multi-modal data set to obtain a standardized multi-physical field sequence comprises:

[0016] The original multi-modal data set is smoothed by a mean filtering method to obtain a denoised data set.

[0017] The timestamps of signals of different modalities in the denoised data set are synchronized, and missing values in the synchronization process are filled by a linear interpolation method to obtain a time series synchronized data set.

[0018] The time series synchronized data set is normalized to map data of different physical dimensions to a unified scale to obtain a standardized multi-physical field sequence.

[0019] Preferably, the spatial feature extraction and temporalization construction of the standardized multi-physical field sequence to obtain a spatial field intensity distribution map comprises:

[0020] The standardized multi-physical field sequence is processed by a convolutional neural network, and multi-dimensional spatial features are extracted to generate a preliminary field intensity distribution map.

[0021] The field intensity deviation value of the preliminary field intensity distribution map is calculated, and the parameters of the convolutional neural network are optimized by a gradient descent method according to the field intensity deviation value to obtain an optimized field intensity distribution map.

[0022] The optimized field strength distribution map is clustered and grouped, spatial correlation between regions in each group is calculated, and principal component analysis is used to reduce the dimensionality of the spatial correlation to obtain a feature set after dimensionality reduction;

[0023] The feature set after dimensionality reduction is stored in a pre-established distributed database, and is organized using time series indexing to obtain a spatial field strength distribution map.

[0024] Preferably, if the field strength distribution in the spatial field strength distribution map exceeds a preset field strength threshold, density clustering and spatial correlation analysis are performed to obtain a potential defect region list, including:

[0025] The spatial field strength distribution map is preprocessed by median filtering, and it is judged whether the preprocessed data value exceeds a preset field strength threshold. The area exceeding the threshold is determined as an abnormal area;

[0026] The abnormal area is divided by density clustering, and areas with high field strength values and dense space are grouped to obtain a high-density scene region set;

[0027] The feature vectors of each region in the high-density scene region set are extracted, and the spatial correlation between regions is calculated using the cosine similarity method to obtain a correlation matrix;

[0028] The multi-dimensional features in the correlation matrix are reduced in dimension by linear discriminant analysis to obtain a potential defect region list.

[0029] Preferably, the spatial data of the potential defect region list is obtained, a three-dimensional structure model inside the connector is constructed, structure analysis and abnormal point positioning are performed, and accurate defect position coordinates are obtained, including:

[0030] According to the potential defect region list, a stereoscopic microscopic three-dimensional scanning is performed on the target area to generate three-dimensional point cloud data;

[0031] The three-dimensional point cloud data is processed by data fusion to integrate multi-modal data such as field strength distribution and temperature signal, and a preliminary three-dimensional structure model of the connector is constructed;

[0032] The preliminary three-dimensional structure model is analyzed and features are extracted to identify key structural features and determine geometric abnormal points therein;

[0033] The geometric abnormal points are calculated and corrected to obtain accurate defect position coordinates.

[0034] Preferably, the multi-time observation values of the accurate defect position coordinates are obtained, time series evolution pattern learning and future trend prediction are performed on the observation values to obtain a future evolution path of the defect, including:

[0035] timestamping the multi-time observation values to obtain an ordered sequence of observation values, calculating difference values of adjacent observation values in the ordered sequence, and collecting all difference values exceeding a preset change threshold to form a defect change increment sequence;

[0036] inputting the defect change increment sequence into a time sequence model for layer-by-layer propagation and learning to obtain a state hidden representation representing the overall evolution pattern of the sequence;

[0037] judging a trend continuation direction and extrapolating according to the state hidden representation to obtain a future evolution path of the defect.

[0038] Preferably, the gradient change of the spatial field strength distribution map is calculated, and the future evolution path is comprehensively modeled to obtain a dynamic field strength update sequence, including:

[0039] calculating the gradient of the spatial field strength distribution map to extract the change rate of the field strength in space to obtain a gradient value sequence;

[0040] judging whether the difference between adjacent values of the gradient value sequence exceeds a preset gradient stability threshold, and when the threshold is exceeded, fusing multiple possible branches of the future evolution path to obtain a fused path branch sequence;

[0041] locating a thermal stress concentration area of the non-uniform region according to the fused path branch sequence and determining a fluctuation sequence of real-time environmental variables;

[0042] performing data fusion on the gradient value sequence, the position data of the thermal stress concentration area, and the fluctuation sequence of the environmental variables to finally generate the dynamic field strength update sequence.

[0043] Preferably, the dynamic field strength update sequence is applied to the real-time acquired field strength distribution data to generate predictive enhanced field strength distribution data;

[0044]

[0045] performing spatial feature extraction on the predictive enhanced field strength distribution data and calculating the change of the features over time to obtain a defect diffusion rate vector;

[0046] performing three-dimensional modeling according to the defect diffusion rate vector and fusing environmental fluctuation and thermal stress data to generate an enhanced three-dimensional coordinate sequence;

[0047] comparing the enhanced three-dimensional coordinate sequence and the defect diffusion rate vector with a preset quality standard, calculating the specific values of the geometric size and diffusion rate exceeding the standard limit, and finally generating a quality detection report.​

[0048] In a second aspect, the present application provides a multi-modal fusion-based FPC connector quality detection system, comprising:

[0049] A multi-modal data acquisition module for acquiring an original multi-modal data set inside the connector, the original multi-modal data set comprising electrical stress, thermal stress and mechanical stress data;

[0050] A data preprocessing module for mean filtering and time series alignment of the original multi-modal data set to obtain a standardized multi-physical field sequence;

[0051] A field intensity map construction module for spatial feature extraction and time series construction of the standardized multi-physical field sequence to obtain a spatial field intensity distribution map;

[0052] A defect detection module for density clustering and spatial correlation analysis if the field intensity distribution in the spatial field intensity distribution map exceeds a preset field intensity threshold to obtain a potential defect area list;

[0053] A three-dimensional defect positioning module for acquiring spatial data of the potential defect area list, constructing a three-dimensional structure model inside the connector, performing structure analysis and abnormal point positioning, and obtaining accurate defect position coordinates;

[0054] An evolution path prediction module for acquiring multi-time observation values of the accurate defect position coordinates, performing time series evolution pattern learning and future trend prediction on the observation values, and obtaining a future evolution path of the defect;

[0055] A dynamic field intensity update module for calculating the gradient change of the spatial field intensity distribution map, combining the future evolution path for comprehensive modeling, and obtaining a dynamic field intensity update sequence;

[0056] A quality assessment and reporting module for performing diffusion rate calculation and quality assessment according to the dynamic field intensity update sequence in combination with real-time acquired field intensity distribution data, and obtaining a final quality detection report.

[0057] Compared with the prior art, the present application has the following beneficial effects:

[0058] (1) The present application acquires multi-modal data such as electrical, thermal and mechanical stress inside the connector, and constructs a unified spatial field intensity distribution map through time series alignment and fusion processing. This method overcomes the limitations of the prior art of analyzing multiple physical fields separately, can reveal the real causes of defects under the action of multiple fields, and thus realizes comprehensive and deep diagnosis of the internal state of the connector, fundamentally improving the identification accuracy of early potential defects.

[0059] (2) The application further constructs a connector internal structure model through a three-dimensional reconstruction algorithm after identifying the field strength abnormal area, and realizes accurate three-dimensional coordinate positioning of defects. This positioning mechanism from macro field strength distribution to micro three-dimensional structure accurately associates abstract performance abnormalities with specific physical forms, solves the problem that the prior art can only provide two-dimensional or fuzzy position information, not only realizes visualization of defects, but also provides accurate geometric input for subsequent evolution prediction.

[0060] (3) The application collects multi-time observation values of the positioned defect coordinates to form a time sequence, and uses time sequence evolution mode learning to predict the future evolution path of the defect. This converts the traditional static "snapshot" detection into dynamic "process" analysis, learns the historical law of defect development, and gives the detection method the scientific inference ability of future trends, effectively solving the core pain point of the lack of foresight of the prior art, and upgrading quality evaluation from post-diagnosis to pre-warning.

[0061] (4) The application comprehensively models the predicted future evolution path and the gradient change of the spatial field strength to generate a dynamic field strength update sequence, and calculates the defect diffusion rate combined with real-time data to complete quality evaluation. This method creatively establishes a closed loop between defect physical form evolution and dynamic response of multiple physical fields, so that the final quality evaluation report has foresight and can quantify the future risk level of defects, providing strong technical support for ensuring long-term reliability of the system. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 is a flowchart of a FPC connector quality detection method based on multi-modal fusion provided by the first embodiment of the application;

[0063] Figure 2 is a structural schematic diagram of a FPC connector quality detection system based on multi-modal fusion provided by the second embodiment of the application. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the application.

[0065] With reference to Figure 1 , the first embodiment of the application provides a FPC connector quality detection method based on multi-modal fusion, including the following steps:

[0066] S11, acquire an original multi-modal data set inside the connector, the original multi-modal data set including electrical stress, thermal stress and mechanical stress data;

[0067] S12, perform mean filtering and time sequence alignment on the original multi-modal data set to obtain a standardized multi-physical field sequence;

[0068] S13, perform spatial feature extraction and time sequence construction on the standardized multi-physical field sequence to obtain a spatial field intensity distribution map;

[0069] S14, if the field intensity distribution in the spatial field intensity distribution map exceeds a preset field intensity threshold, perform density clustering and spatial correlation analysis to obtain a potential defect area list;

[0070] S15, acquire spatial data of the potential defect area list, construct a three-dimensional structure model inside the connector, perform structure analysis and abnormal point positioning, and obtain accurate defect position coordinates;

[0071] S16, acquire multi-time observation values of the accurate defect position coordinates, perform time sequence evolution mode learning and future trend prediction on the observation values, and obtain a future evolution path of the defect;

[0072] S17, calculate gradient changes of the spatial field intensity distribution map, combine the future evolution path to perform comprehensive modeling, and obtain a dynamic field intensity update sequence;

[0073] S18, according to the dynamic field intensity update sequence, combine real-time acquired field intensity distribution data to perform diffusion rate calculation and quality assessment, and obtain a final quality detection report.

[0074] In step S11, an original multi-modal data set inside the connector is acquired, and the original multi-modal data set includes electrical stress, thermal stress and mechanical stress data.

[0075] It should be noted that the core of step S11 is to perform synchronous and high-fidelity data acquisition on multiple physical dimensions of key nodes inside the FPC connector through an integrated multi-sensor array. The array is not a simple collection of sensors, but a system that works cooperatively, aiming to capture weak signal responses caused by potential defects and coupled in different physical fields, so as to construct an original multi-modal data set that can fully represent the health status of the connector.

[0076] Specifically, the acquisition of the original multi-modal data set includes the following contents:

[0077] The acquisition of electrical stress data is achieved by deploying non-contact Hall Effect Current Sensors near critical conductive paths of the connector. These sensors, based on the Hall Effect principle, accurately calculate the amount of current flowing through by measuring changes in the magnetic field strength around the conductor. For example, continuous monitoring of the current at a sampling frequency of 10 kHz generates a current time series reflecting load fluctuations, transient overcurrents, or changes in contact resistance, quantified in amperes (A).

[0078] The acquisition of thermal stress data is achieved by attaching miniature K-type Thermocouples to hot spots such as connector plug-in interfaces and crimped terminals. These thermocouples utilize the Seebeck effect to convert the weak voltage generated by temperature differences at different metal contacts into accurate temperature readings. For example, temperature collection at a sampling frequency of 20 Hz generates a temperature time series reflecting Joule heat accumulation, poor heat dissipation, or environmental temperature effects, quantified in degrees Celsius.

[0079] The acquisition of mechanical stress data is achieved by attaching resistive strain gauges to stress concentration areas at the root of the connector housing or FPC ribbon. The core principle of these strain gauges is the piezoresistive effect of metals, whose resistance value changes linearly with the physical deformation (stretching or compression) of the base material. For example, by measuring the change in resistance value through a high-precision bridge circuit at a sampling frequency of 5 kHz, it is converted into a dimensionless micro-strain time series to represent the structural response of the connector under vibration, impact, or plug-in operation.

[0080] In step S12, the original multi-modal data set is subjected to mean filtering and time series alignment to obtain a standardized multi-physical field sequence. It should be noted that in the first embodiment of the present application, this processing step specifically includes steps S121 to S123:

[0081] S121, the original multi-modal data set is subjected to mean filtering method for smoothing processing to obtain a denoised data set;

[0082] S122, the signals of different modalities in the denoised data set are subjected to timestamp synchronization processing, and the linear interpolation method is used to fill in the missing values in the synchronization process to obtain a time series synchronized data set;

[0083] S123, the time series synchronized data set is subjected to normalization processing to map data of different physical dimensions to a unified scale to obtain a standardized multi-physical field sequence.

[0084] In step S121, a mean filter method is used to smooth the original multi-modal data set to obtain a denoised data set. Specifically, this step aims to suppress high-frequency noise and transient interference introduced during acquisition. In this embodiment, a moving average filter is used to achieve this function. The core principle of this filter is to take the arithmetic mean of all data points in the window as the new value of the center point. For example, the size of the sliding window N can be set to 5 sampling points. It should be noted that the value of the window size N is determined based on experimental statistical analysis on historical noise data sets. By testing the smoothing effect of different window values (e.g., N=3, 5, 7) on typical noise signals, and evaluating their waveform fidelity to the original effective signal (such as electrical stress spikes), an optimal value (e.g., N=5) is finally selected that can maximize the signal-to-noise ratio while keeping the signal distortion within a predetermined range. When there are abnormal spikes in the electrical stress data due to electromagnetic crosstalk, this process can effectively smooth the spikes while preserving the low-frequency trend characteristics of the signal.

[0085] It should be noted that the start of the filtering process can be triggered by a preset noise level threshold. This threshold is set by statistical analysis of noise data of similar connectors under typical working conditions, aiming to balance the denoising effect and signal fidelity. For example, when the signal-to-noise ratio (SNR) of the monitored signal is lower than 15 decibels, the mean filter is started. In a preferred scheme, if the noise level after mean filtering is still higher than a second threshold (e.g., the signal-to-noise ratio is still lower than 10 decibels), a median filter (Median Filter) can be further enabled for secondary processing to deal with stronger impulse noise.

[0086] In step S122, the signals of different modalities in the denoised data set are time-stamped synchronized, and linear interpolation method is used to fill in the missing values in the synchronization process to obtain a time-synchronized data set. Specifically, this step aims to solve the problem of inconsistent sampling frequencies caused by different physical characteristics of sensors. In this embodiment, resampling technology is used to unify all modal time series data to a common time reference grid. The frequency of this common grid is set according to the Nyquist-Shannon sampling theorem, which should be at least twice the highest effective frequency in all input signals. For example, if the mechanical stress signal contains vibration information up to 100 Hz, the frequency of the common time grid can be set to 250 Hz.

[0087] In the resampling process, for the original sampling rate lower sequence (for example, 10 hertz temperature sequence), on the vacancy time point of the new time grid, the linear interpolation algorithm is used to estimate its value. The principle of this algorithm is based on two known, adjacent data points, create a linear function between them, and calculate the function value of any intermediate point according to the function, the calculation process is as follows:

[0088] First, locate the anchor point: for any target time point that needs to be interpolated on the unified time grid , first locate the two nearest, known real data points and in the original low-frequency time sequence, which are located just before and after .

[0089] Then, calculate the rate of change: calculate the linear rate of change (i.e. slope) between the two known "anchor points" , the calculation formula is: .

[0090] Finally, calculate the interpolation: finally, based on the point-slope equation of a straight line, calculate the corresponding interpolation of the target time point , the calculation formula is: . By repeating the above process for all missing points, a complete low-frequency data stream that is completely aligned in time with the high-frequency data stream can be generated.

[0091] In step S123, the time series synchronized data set is normalized to map data of different physical dimensions to a unified scale to obtain a standardized multi-physical field sequence. Specifically, this step aims to eliminate the scale difference between different physical dimensions (such as amperes of current, degrees Celsius of temperature, and micro-strains of strain), and avoid the dominance or neglect of model parameter updates due to the large or small value range of features in subsequent machine learning model training. This embodiment adopts the Min-Max normalization method. The core principle of this method is to strictly map the original data in any interval to the interval by linear transformation. Its transformation function is:

[0092] ;

[0093] where X is the original data point, and ​respectively. For example, a temperature data sequence with a range of [25℃, 85℃] will be normalized as (65-25) / (85-25)≈0.67 for a reading of 65℃. It is to be noted that the mapping of data to the interval [0, 1] is a well-known technique in the field of deep learning, which can effectively improve the convergence speed and stability of the model based on the gradient descent algorithm.

[0094] In step S13, spatial feature extraction and time series construction are performed on the standardized multi-physical field sequence to obtain a spatial field intensity distribution map. It is to be noted that in the first embodiment of the present application, this processing step specifically includes steps S131 to S134:

[0095] S131, the standardized multi-physical field sequence is processed using a convolutional neural network, and multi-dimensional spatial features are extracted to generate a preliminary field intensity distribution map;

[0096] S132, the field intensity deviation value of the preliminary field intensity distribution map is calculated, and the parameters of the convolutional neural network are optimized using the gradient descent method according to the field intensity deviation value to obtain an optimized field intensity distribution map;

[0097] S133, the optimized field intensity distribution map is clustered and grouped, the spatial correlation between each group region is calculated, and the principal component analysis method is used to reduce the dimensionality of the spatial correlation to obtain a reduced feature set;

[0098] S134, the reduced feature set is stored in a pre-established distributed database, and is organized using a time series index to obtain a spatial field intensity distribution map.

[0099] In step S131, the standardized multi-physical field sequence is processed using a convolutional neural network to extract multi-dimensional spatial features and generate a preliminary field intensity distribution map. Specifically, the standardized multi-physical field sequence is first constructed into a three-dimensional tensor with dimensions such as [N, M, 3], where N x M represents the spatial grid resolution of the connector internal monitoring area, and 3 represents the three physical field channels of electricity, heat, and mechanics. The convolutional neural network (CNN) used in this embodiment is a hybrid structure containing two convolutional layers and two fully connected layers. The convolutional layer of the network uses a 3 x 3 convolution kernel (the number of convolution kernels can be 16-64) and a rectified linear unit (ReLU) as the activation function. The core principle is to use the convolution kernel as a learnable feature filter to automatically extract local spatial features such as field intensity boundaries and gradient discontinuities by sliding in space. The fully connected layer is responsible for nonlinearly combining the extracted local features, and finally outputs a single-channel preliminary field intensity distribution map with the same size as the input grid N x M.

[0100] It should be noted that the CNN model is pre-trained through supervised learning. The training data set comes from simulation data obtained by finite element analysis (FEA) of the same type of FPC connector under various working conditions. The simulation input (multi-physical field excitation) is used as the model input, and the known field intensity real distribution map obtained by simulation is used as the label (Ground Truth). The mean squared error (MSE) is used as the loss function, and the Adam optimizer is used for iterative training. The learning rate is usually set between 0.001 and 0.0001, and the model converges until the model converges.

[0101] In step S132, the field strength deviation value of the preliminary field strength distribution map is calculated, and the parameters of the convolutional neural network are optimized according to the field strength deviation value by using a gradient descent method to obtain an optimized field strength distribution map. Specifically, in order to make the pre-trained model better adapt to the real-time working condition of the current specific connector, online optimization is required. First, the mean square error (MSE) between the preliminary field strength distribution map and the sparse real sensor readings is calculated as the field strength deviation value. Then, if the deviation value exceeds a preset "model optimization trigger threshold" (which can be determined by cross-validation, ROC curve analysis or historical fault data statistics, and is usually set in the MSE range of 0.1-0.3, which can be adjusted according to the model accuracy requirement), the gradient descent (Gradient Descent) algorithm is started. The core principle of this algorithm is to adjust the weight parameters of the CNN model (especially the fully connected layer) in the direction of the fastest gradient descent of the loss function with a preset learning rate (learning rate, for example 0.001).

[0102] In step S133, the optimized field strength distribution map is clustered and grouped, the spatial correlation between each grouped region is calculated, and the principal component analysis method is used to reduce the dimension of the spatial correlation to obtain a reduced feature set. Specifically, first, the K-means clustering (K-Means Clustering) algorithm is used to divide the region of the optimized field strength distribution map. This algorithm divides all pixel points in the map into a preset K clusters according to their field strength values through iterative calculation, so that the variance of points within the cluster is minimized. For example, by setting K=3, the field strength map can be automatically divided into three regions of high, medium and low levels. Then, the Euclidean distance between the centroids of each region is calculated to construct a relationship matrix representing the spatial proximity relationship of each field strength region.

[0103] Finally, the relationship matrix is subjected to principal component analysis (Principal Component Analysis, PCA). The core principle of PCA is to project high-dimensional correlated feature vectors onto a set of low-dimensional orthogonal coordinate systems through linear transformation, and these new coordinate axes are called principal components. This embodiment retains the first k principal components (for example, k=3) that can explain more than 90% of the cumulative variance contribution rate of the original data. It should be noted that the threshold of 90% is an engineering experience value that balances the completeness of information retention and data compression efficiency, and those skilled in the art can adjust it within the range of 85% to 98% according to the specific accuracy requirement.

[0104] In step S134, the reduced dimension feature set is stored in a pre-established distributed database and organized using time series indexing to obtain a spatial field intensity distribution map. Specifically, to meet the efficient storage and query requirements of massive time series data, the present embodiment uses a distributed database based on time series (for example, InfluxDB). The database uses sharding technology to horizontally expand data to multiple physical nodes according to time range (for example, one shard per 24 hours). At the same time, an efficient index structure such as B+ tree is established for the data primary key (timestamp). The core advantage of this architecture is that when performing a time range query (for example, retrieving all field intensity features in the past hour), the database engine can directly locate the corresponding data shard and index location, avoiding full table scanning, thereby achieving millisecond-level query response. Finally, this three-dimensional feature dataset organized by timestamp and efficiently queryable constitutes a dynamic, traceable spatial field intensity distribution map sequence.

[0105] In step S14, if the field intensity distribution in the spatial field intensity distribution map exceeds a preset field intensity threshold, density clustering and spatial correlation analysis are performed to obtain a potential defect area list. It should be noted that in the first embodiment of the present application, this processing step specifically includes steps S141 to S144:

[0106] S141, median filtering preprocessing is performed on the spatial field intensity distribution map, and it is determined whether the data value after preprocessing exceeds a preset field intensity threshold. Regions exceeding the threshold are determined as abnormal regions;

[0107] S142, density clustering is used to divide the abnormal regions, and regions with high field intensity values and dense in space are grouped to obtain a high-density scene region set;

[0108] S143, feature vectors of each region in the high-density scene region set are extracted, and the spatial correlation between regions is calculated using the cosine similarity method to obtain a correlation matrix;

[0109] S144, linear discriminant analysis method is used to reduce the dimension of the multi-dimensional features in the correlation matrix to obtain a potential defect area list.

[0110] In step S141, the spatial field intensity distribution map is median filtered and preprocessed, and it is determined whether the data value after preprocessing exceeds a preset field intensity threshold value, and the area exceeding the threshold value is determined as an abnormal area. Specifically, first, the input field intensity distribution map is median filtered, and the filter replaces the value of each data point with the median value of the numerical values in its neighborhood window, effectively eliminating isolated noise points. For example, for the sequence [10, 12, 50, 11, 13], the median value 11 will be used to replace the abnormal value 50. Subsequently, the numerical value of each point in the filtered map is compared with a preset defect activation threshold value. The threshold value (for example, 15V / m) is set based on statistical analysis of the field intensity data of historical similar connectors in the normal aging process, and the 99% percentile of the distribution is selected, so as to ensure that only statistically significant abnormalities are marked.

[0111] In step S142, the abnormal areas are divided by using density clustering, the areas with high field intensity values and high spatial density are grouped, and a high-density scene area set is obtained. Specifically, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm is used in this embodiment. The core principle of DBSCAN is not based on the distance to the cluster center, but on the "density reachability" of the area. It defines the neighborhood by two key parameters: neighborhood radius and minimum neighborhood sample number MinPts of core points. The algorithm starts from an unvisited core point, and recursively merges all density-reachable points of the core point into a cluster. For example, the neighborhood radius is set to 1.5mm, and the minimum sample number MinPts is set to 10 data points. The values of these two parameters are based on the physical design specifications of the connector and the spatial resolution of the sensor, and correspond to a minimum defect size that is considered meaningful in a physical sense. DBSCAN can discover defect clusters of any shape and automatically identify sparse, density-unqualified abnormal points as noise, which is very suitable for irregular defect identification.

[0112] In step S143, the feature vectors of the areas in the high-density scene area set are extracted, and the spatial correlation between the areas is calculated by using the cosine similarity method, and a correlation matrix is obtained. Specifically, first, a multi-dimensional feature vector is constructed for each high-density area (i.e. defect cluster) obtained in S142. The vector includes quantitative indicators describing the characteristics of the area, for example, the vector can be [average field intensity value, peak temperature, area volume, geometric irregularity]. Subsequently, the cosine similarity (Cosine Similarity) is used to calculate the feature vectors and The principle is to calculate the cosine value of the angle between two vectors in a multi-dimensional space:

[0113] ;

[0114] The result of this value ranges between [-1, 1], and the closer the value is to 1, the more similar the intrinsic physical characteristics of the two defect clusters are, and they are likely to be caused by the same root cause (e.g., the extension of the same crack). For example, the feature vectors of two regions are [18, 10, 5] and [20, 12, 6] respectively, and the calculated cosine similarity can be as high as 0.98.

[0115] In step S144, a linear discriminant analysis method is used to reduce the dimensionality of the multi-dimensional features in the correlation matrix to obtain a list of potential defect regions. Specifically, the correlation matrix generated in S143 and the feature vectors of each region are combined to form a high-dimensional feature space. Linear Discriminant Analysis (LDA) is used to reduce the dimensionality of this space. LDA is a supervised dimensionality reduction technique, and its core principle is to find a projection direction that maximizes the distance between samples of different classes (e.g., "high-risk cracks" and "general hot spots" labeled according to historical data) and minimizes the variance of samples within the same class. Through this method, high-dimensional features (e.g., more than 10 feature dimensions) can be projected into a low-dimensional space (e.g., 2D) that best reflects the risk level of defects, and finally a clear, risk level ordered list of potential defect regions is generated. The linear discriminant analysis can be used according to whether there is labeled data in practical application, if there is labeled information of historical defect types, using LDA can effectively extract discriminative features; if there is no label, it can be replaced by Principal Component Analysis (PCA) or other unsupervised dimensionality reduction methods to achieve feature compression and visualization.

[0116] In step S15, the spatial data of the list of potential defect regions is obtained, a three-dimensional structure model of the connector is constructed, and structure analysis and abnormal point positioning are performed to obtain the accurate defect position coordinates. It should be noted that in the first embodiment of the present application, this processing step specifically includes steps S151 to S154:

[0117] S151, according to the list of potential defect regions, performing stereomicroscopic three-dimensional scanning on the target region to generate three-dimensional point cloud data;

[0118] S152, performing data fusion processing on the three-dimensional point cloud data, integrating multi-modal data such as field intensity distribution and temperature signal, and constructing a preliminary three-dimensional structure model of the connector;

[0119] S153, structure analysis and feature extraction are performed on the preliminary three-dimensional structure model, key structural features are identified, and geometric abnormal points are determined;

[0120] S154, coordinate calculation and correction are performed on the geometric abnormal points to obtain accurate defect position coordinates.

[0121] In step S151, according to the list of potential defect areas, stereomicroscopic three-dimensional scanning is performed on the target area to generate three-dimensional point cloud data. Specifically, the core of this step is to obtain high-fidelity three-dimensional geometric morphology of the defect area. This embodiment adopts scanning technology based on the principle of binocular stereo vision. This technology simulates human visual parallax to synchronously collect two-dimensional images of the target area from two or more different angles, and through calculating the pixel displacement (parallax) of the same point in different images, the three-dimensional space coordinates of each pixel point are reconstructed by using the triangulation principle. For example, for a welding spot defect area in the list, the scanning system can scan at a spatial resolution of 0.1 mm, and finally generate a three-dimensional point cloud data set composed of tens of thousands of (x, y, z) coordinate points.

[0122] In step S152, the three-dimensional point cloud data is subjected to data fusion processing, and multi-modal data such as field intensity distribution and temperature signals are integrated to construct a preliminary three-dimensional structure model inside the connector. Specifically, this step aims to accurately register and fuse the pure geometric model (target point cloud) obtained in S151 with the physical field data (source point cloud) with spatial coordinates obtained in the previous steps. This embodiment uses the Iterative Closest Point (ICP) algorithm to achieve accurate spatial alignment.

[0123] The iteration process of the ICP algorithm is as follows: first, point pair association, for each point in the source point cloud, find the closest point in the target point cloud to form an initial matching point pair. Then, transformation solving, construct a loss function aimed at minimizing the mean square distance error between all matching point pairs, and solve the best rigid transformation parameters that minimize the loss function, i.e. a rotation matrix R and a translation vector t, by using numerical optimization methods such as Singular Value Decomposition (SVD). Next, point cloud transformation, apply the rotation matrix R and translation vector t obtained to the entire source point cloud. Finally, iteration termination judgment, calculate the mean square distance error of the transformed source point cloud and the target point cloud, if the error is less than the preset convergence threshold or the number of iterations reaches the upper limit (e.g. 100 times), the algorithm terminates and outputs the final transformation matrix.

[0124] After the alignment of the two point cloud coordinate systems through the ICP algorithm, the weighted fusion of the multi-modal data is performed. For example, the comprehensive attribute of any point p in the model The comprehensive attribute can be calculated by the following formula:

[0125]

[0126] wherein are the electrical, thermal, and mechanical stress values of the point, are the corresponding weight coefficients (for example, , is the sum of the weights). It should be noted that the setting of the weight coefficients is based on statistical analysis of historical failure samples or automatic learning through a machine learning model (such as a neural network), aiming to highlight the physical field information most strongly associated with the type of defect, rather than being fixed.

[0127] In step S153, the preliminary three-dimensional structure model is subjected to structure analysis and feature extraction, key structural features are identified, and geometric abnormal points are determined. Specifically, this step automatically "understands" the geometric shape of the three-dimensional model through an algorithm. In this embodiment, curvature analysis based on point cloud normal vectors is used to identify geometric abnormalities. By calculating the local curvature of each point on the model surface, the mutation region of the geometric shape can be effectively identified. The setting of this curvature threshold is based on statistical analysis of the geometric features of good samples, and the upper limit of the normal range is selected, for example, the 99.5% quantile. When the calculated curvature of a point in the model exceeds this threshold, the point is marked as a geometric abnormal point. For example, through this method, the curvature abnormal region caused by micro-cracks can be accurately identified from the surface of a weld point with a diameter of 2.5 mm.

[0128] In step S154, the coordinates of the geometric abnormal points are calculated and corrected to obtain the precise defect position coordinates. Specifically, this step aims to perform the final precise calibration of the abnormal point coordinates identified in S153. In this embodiment, at least three fiducial markers pre-set on the connector are introduced, and the initial coordinates of the abnormal points obtained during the scanning in S151 are subjected to global coordinate transformation and optimization. This process calculates the optimal estimated coordinates of the abnormal points by minimizing the re-projection error of all points in the three-dimensional space. For example, an abnormal point with an initial coordinate of (x=15.2 mm, y=8.7 mm, z=3.1 mm) is corrected by the fiducial markers, and the final output of the precise defect position coordinates is (x=15.02 mm, y=8.75 mm, z=3.11 mm).

[0129] ​In step S16, multi-time observation values of the accurate defect position coordinates are obtained, time sequence evolution mode learning and future trend prediction are performed on the observation values, and a future evolution path of the defect is obtained. It should be noted that in the first embodiment of the present application, this processing step specifically includes steps S161 to S163:

[0130] S161, time stamp sorting is performed on the multi-time observation values to obtain an ordered sequence of observation values, difference values of adjacent observation values in the ordered sequence are calculated, and all difference values exceeding a preset change threshold are collected to form a defect change increment sequence;

[0131] S162, the defect change increment sequence is input into a time sequence model for layer-by-layer propagation and learning to obtain a state hidden representation representing the overall evolution mode of the sequence;

[0132] S163, a trend continuation direction is judged according to the state hidden representation and extrapolation is performed to obtain a future evolution path of the defect.

[0133] In step S161, the multi-time observation values are time stamp sorted to obtain an ordered sequence of observation values, difference values of adjacent observation values in the ordered sequence are calculated, and all difference values exceeding a preset change threshold are collected to form a defect change increment sequence. Specifically, this step aims to convert the original, absolute observation value sequence into a sparse, incremental sequence that better reflects dynamic changes. First, the multi-time observation values (such as the curvature of the defect point) collected within, for example, 24 consecutive hours are arranged in ascending order of time stamp. Then, the difference between each two adjacent observation values in the ordered sequence is calculated, and the absolute value of the difference is compared with a preset "evolution significance threshold". The threshold (for example, a curvature change of 0.02 mm) is set based on material fatigue theory and experimental statistics or by observing the minimum significant change in defect expansion in accelerated life tests to ensure its engineering applicability and statistical significance, which represents the minimum change amount sufficient to exclude measurement noise and mark the real evolution of the physical structure. All difference values exceeding the threshold (retaining their signs) are collected to form the final defect change increment sequence.

[0134] In step S162, the defect change increment sequence is input into a time sequence model for layer-by-layer propagation and learning to obtain a state hidden representation representing the overall evolution mode of the sequence. Specifically, the time sequence model used in this embodiment is a stacked long short-term memory (LSTM). Specifically, the network is composed of an input layer, two stacked LSTM layers, and an output layer. The input layer receives the defect change increment sequence generated in S161. The first LSTM layer includes 64 hidden units, and the second LSTM layer includes 32 hidden units. The core of the LSTM unit is the "gating mechanism" (input gate, forget gate, and output gate) inside it, which enables it to selectively remember long-term dependencies in the sequence (e.g., cumulative damage effect of defects) while forgetting unimportant short-term fluctuations. The final output layer outputs a fixed-dimensional vector, i.e., the state hidden representation. The LSTM model is pre-trained through supervised learning. The training data set is derived from a large amount of historical full-life monitoring data of similar connectors. The defect change increment sequence of the historical sample is used as the model input, and its known, true subsequent evolution path is used as the label. The root mean square error (RMSE) is used as the loss function, and the Adam optimizer is used for iterative training, with a learning rate usually set between 0.001 and 0.0001, until the model can accurately output a hidden state representing the future trend based on the input historical change pattern.

[0135] In step S163, the trend continuation direction is determined based on the state hidden representation and extrapolated to obtain the future evolution path of the defect. Specifically, the state hidden representation output by S162 is a highly condensed feature vector, and the direction and norm of the vector contain the cumulative trend of defect evolution. In this embodiment, a fully connected layer (also known as a prediction head) is used to decode the state hidden representation. The prediction head is trained to map the input hidden representation to a specific future path description. For example, if the dimensions representing "positive growth" and "acceleration" in the hidden representation vector have higher weight values, the decoder will output a prediction path of accelerated expansion, such as: "It is predicted that the defect curvature will increase from 0.12 mm to 0.18 mm in the next 12 hours, and will mainly expand along the Z-axis direction."

[0136] In step S17, the gradient change of the spatial field strength distribution map is calculated, and a comprehensive model is established in combination with the future evolution path to obtain a dynamic field strength update sequence. It should be noted that in the first embodiment of the present application, this processing step specifically includes steps S171 to S174:

[0137] S171, calculating the gradient of the spatial field strength distribution map to extract the rate of change of the field strength in space to obtain a gradient value sequence;

[0138] S172, judging whether the difference between adjacent values of the gradient value sequence exceeds a preset gradient stability threshold, and when the threshold is exceeded, fusing multiple possible branches of the future evolution path to obtain a fused path branch sequence;

[0139] S173, locating a thermal stress concentration area of the non-uniform area according to the fused path branch sequence, and determining a fluctuation sequence of the real-time environmental variable;

[0140] S174, performing data fusion on the gradient value sequence, the position data of the thermal stress concentration area, and the fluctuation sequence of the environmental variable to finally generate the dynamic field strength update sequence.

[0141] In step S171, the gradient of the spatial field strength distribution map is calculated to extract the rate of change of the field strength in space, and a gradient value sequence is obtained. Specifically, this step is implemented by a three-dimensional Sobel operator. The Sobel operator is a discrete differential operator, and its core principle is to approximate the partial derivative of each data point in three orthogonal directions in space by performing convolution operation on the field strength distribution map and a 3x3x3 convolution kernel. Finally, the gradient of each point is calculated as the norm of its gradient vector, for example, 0.22V / m / mm, which represents the degree of change of the field strength in space.

[0142] In step S172, it is judged whether the difference between adjacent values of the gradient value sequence exceeds a preset gradient stability threshold, and when the threshold is exceeded, multiple possible branches of the future evolution path are fused to obtain a fused path branch sequence. Specifically, the first-order difference of the gradient value time sequence obtained in S171 is first calculated. Then, the difference value is compared with the gradient stability threshold. The threshold (for example, 0.05V / m / mm) is determined based on the statistical analysis of the natural fluctuation of the field strength gradient of the connector in the stable working state for 100 hours of continuous operation, and is selected based on the 98% quantile of the distribution. When the difference value exceeds this threshold, it indicates that the field strength change enters an unstable state, and at this time, path fusion is enabled. The fusion is performed by weighting and averaging the multiple predicted path branches output by S16 according to their respective confidence levels, and integrating them into a single fused path branch sequence with the highest expected value.

[0143] In step S173, the thermal stress concentration area of the non-uniform area is located according to the fusion path branch sequence, and the fluctuation sequence of the real-time environmental variable is determined. Specifically, this step processes two inputs in parallel: first, the fused defect propagation path is taken as input, and simulation calculation is performed in a pre-constructed connector thermal coupling model based on the finite element method (FEM) (the FEM model is constructed based on the material properties and structural design parameters of the connector, and is verified by experimental data such as thermal imaging), so as to locate the three-dimensional coordinates (for example, peak stress 2.5 MPa) of the new thermal stress concentration area caused by the path. Second, the real-time collected environmental variable (such as humidity, vibration) sequence is processed by a pre-trained LSTM network, the fluctuation mode is analyzed, and a state hidden representation vector (for example, [0.08, 0.12]) that can represent the short-term change trend is output.

[0144] In step S174, the gradient value sequence, the position data of the thermal stress concentration area, and the fluctuation sequence of the environmental variable are fused to finally generate the dynamic field strength update sequence. Specifically, this step performs final nonlinear fusion on multiple feature streams generated in the previous steps by a multi-layer perceptron (MLP) fusion network. The input layer of the MLP network receives a concatenated feature vector, which at least includes the gradient value at the current time, the three-dimensional coordinates of the thermal stress concentration area, and the state hidden representation of the environmental variable. After training, the output layer of the network can generate an accurate scalar value, which is the predicted update value of the field strength at the next time. For example, according to the fusion analysis, the field strength value of 4.8 V / m at the current time is updated to the predicted value of 4.6 V / m at the next time. By repeating this process for all key points in the field strength map, the final dynamic field strength update sequence is generated.

[0145] In step S18, according to the dynamic field strength update sequence, the diffusion rate calculation and quality assessment are performed in combination with the real-time acquired field strength distribution data to obtain a final quality detection report. It should be noted that in the first embodiment of the present application, this processing step specifically includes steps S181 to S184:

[0146] S181, applying the dynamic field strength update sequence to the real-time acquired field strength distribution data to generate predictive enhanced field strength distribution data;

[0147] S182, performing spatial feature extraction on the predictive enhanced field strength distribution data and calculating the change of the feature over time to obtain a defect diffusion rate vector;

[0148] S183, three-dimensional modeling according to the defect diffusion rate vector, and merging environmental fluctuation and thermal stress data to generate enhanced three-dimensional coordinate sequence;

[0149] S184, comparing the enhanced three-dimensional coordinate sequence with the defect diffusion rate vector with the preset quality standard, calculating the specific value of the geometric size and the diffusion rate exceeding the standard limit, and finally generating a quality detection report.

[0150] In step S181, the dynamic field strength update sequence is applied to the real-time acquired field strength distribution data to generate predictive enhanced field strength distribution data. Specifically, this step fuses prediction and observation through a Kalman Filter framework. Among them, the dynamic field strength update sequence output by S17 is used as the prediction step of this framework, which provides a priori estimate of the current field strength state; while the real-time acquired field strength distribution data is used as the update step, which provides the true measurement of the current state. The Kalman filter optimally combines the predicted value and the measured value according to their respective uncertainties (quantified by the process noise covariance and the measurement noise covariance matrix) by calculating the Kalman gain, and finally outputs a posteriori estimate, i.e. the predictive enhanced field strength distribution data. This data is closer to the true state than pure prediction or pure measurement; the process noise covariance matrix can be initialized by the covariance estimation of historical prediction errors; the measurement noise covariance matrix can be determined by sensor calibration data; in a preferred scheme, adaptive Kalman filtering (AKF) technology can be used for real-time adjustment.

[0151] In step S182, spatial feature extraction is performed on the predictive enhanced field strength distribution data, and the change of the feature with time is calculated to obtain the defect diffusion rate vector. Specifically, this step first uses the Isocontour Analysis algorithm to extract the specific field strength contour representing the defect boundary on the enhanced field strength map generated in S181. Then, by comparing the change of the position of the contour centroid at two consecutive time steps (for example, the time interval is 100 milliseconds), the displacement vector in space is calculated. The defect diffusion rate vector is obtained by dividing the displacement vector by the time interval.

[0152] In step S183, a three-dimensional model is built according to the defect diffusion rate vector, and environmental fluctuations and thermal stress data are fused to generate an enhanced three-dimensional coordinate sequence. Specifically, this step constructs a three-dimensional motion trajectory of the defect core area by time integrating the instantaneous rate vector obtained in S182. The initial point of the trajectory is derived from the accurate defect position coordinates determined in S15. To further improve the accuracy of the trajectory, the model also fuses the environmental fluctuation trend and thermal stress concentration zone position data analyzed in S173 to fine-tune the trajectory to compensate for the disturbance of external factors on the defect propagation path, and finally generates the enhanced three-dimensional coordinate sequence.

[0153] In step S184, the enhanced three-dimensional coordinate sequence and the defect diffusion rate vector are compared with a predetermined quality standard, the specific values of the geometric size and diffusion rate that exceed the standard limit are calculated, and finally a quality detection report is generated. Specifically, the quality standard is a pre-defined multi-dimensional safety envelope. The basis for setting this envelope is the design specification of the connector, the material fatigue limit, and the industry safety standard (e.g., GJB standard for the aviation industry). The evaluation process includes geometric evaluation and dynamic evaluation. In geometric evaluation, it is determined whether the enhanced three-dimensional coordinate sequence generated in S183 has invaded the "no entry" area defined in the safety envelope, and the specific value of the size exceeding the allowed maximum size (e.g., 0.5 mm) is calculated. In dynamic evaluation, it is determined whether the modulus of the defect diffusion rate vector obtained in S182 exceeds the "maximum allowed expansion rate" (e.g., 0.09 mm / s) defined in the safety envelope, and the specific value of the excess rate is calculated. Finally, these quantified deviation values are written together with the corresponding risk level to generate the final quality detection report.

[0154] Referring to Figure 2 , the second embodiment of the present application provides a multi-modal fusion-based FPC connector quality detection system, comprising:

[0155] A multi-modal data acquisition module for acquiring an original multi-modal data set inside the connector, the original multi-modal data set including electrical stress, thermal stress, and mechanical stress data;

[0156] A data preprocessing module for mean filtering and time series alignment of the original multi-modal data set to obtain a standardized multi-physical field sequence;

[0157] A field strength map construction module for spatial feature extraction and time series construction of the standardized multi-physical field sequence to obtain a spatial field strength distribution map;

[0158] A defect detection module for, if the field strength distribution in the spatial field strength distribution map exceeds a pre-set field strength threshold, performing density clustering and spatial correlation analysis to obtain a list of potential defect areas;

[0159] a three-dimensional defect positioning module configured to acquire spatial data of the potential defect area list, construct a three-dimensional structure model of the connector, perform structural analysis and abnormal point positioning, and obtain accurate defect position coordinates;

[0160] an evolution path prediction module configured to acquire multi-time observation values of the accurate defect position coordinates, perform time series evolution mode learning and future trend prediction on the observation values, and obtain a future evolution path of the defect;

[0161] a dynamic field strength update module configured to calculate gradient changes of the spatial field strength distribution map, combine the future evolution path to perform comprehensive modeling, and obtain a dynamic field strength update sequence;

[0162] a quality assessment and reporting module configured to perform diffusion rate calculation and quality assessment according to the dynamic field strength update sequence and in combination with real-time acquired field strength distribution data, and obtain a final quality detection report.

[0163] It should be noted that the FPC connector quality detection system based on multi-modal fusion provided by the embodiments of the present application is used to execute all process steps of the FPC connector quality detection method based on multi-modal fusion of the above embodiments, and the working principles and beneficial effects of the two are one-to-one correspondence, thus not being repeated.

[0164] The embodiments of the present application further provide an electronic device. The electronic device includes a processor, a memory, and a computer program, such as a dynamic field strength update program, stored in the memory and executable on the processor. The processor implements the steps in each of the above FPC connector quality detection methods based on multi-modal fusion when executing the computer program, such as the step S11 shown in the figure. Figure 1 Alternatively, the processor implements the functions of each module / unit in each of the above device embodiments when executing the computer program, such as the three-dimensional defect positioning module.

[0165] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the electronic device.

[0166] The electronic device can be a computing device such as a desktop computer, a notebook computer, a palm computer, a smart tablet, etc. The electronic device can include, but is not limited to, a processor, a memory. Those skilled in the art can understand that the above components are only examples of the electronic device and do not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the above, or combine certain components, or different components, for example, the electronic device can also include an input / output device, a network access device, a bus, etc.

[0167] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor is the control center of the electronic device, and connects various parts of the electronic device through various interfaces and lines.

[0168] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the electronic device by running or executing the computer program and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0169] The modules / units integrated in the electronic device, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or device, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. that can carry the computer program code. It should be noted that the contents included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0170] It should be noted that the above-described device embodiments are only schematic, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. In addition, the connection relationship between the modules in the device embodiment provided by the present application indicates that there is a communication connection between them, which can be realized as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0171] The above-described specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above-described specific embodiments are only for the specific embodiments of the present application and do not limit the protection scope of the present application. It is particularly pointed out that any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A multi-modal fusion-based FPC connector quality detection method, characterized in that, The method comprises the following steps: acquiring an original multi-modal data set inside a connector, the original multi-modal data set comprising electrical stress, thermal stress and mechanical stress data; performing mean filtering and time series alignment on the original multi-modal data set to obtain a standardized multi-physical field sequence; performing spatial feature extraction and time series construction on the standardized multi-physical field sequence to obtain a spatial field intensity distribution map; if the field intensity distribution in the spatial field intensity distribution map exceeds a preset field intensity threshold, performing density clustering and spatial correlation analysis to obtain a potential defect area list; acquiring spatial data of the potential defect area list, constructing a three-dimensional structure model inside the connector, performing structure analysis and abnormal point positioning to obtain accurate defect position coordinates; acquiring multi-time observation values of the accurate defect position coordinates, performing time series evolution mode learning and future trend prediction on the observation values to obtain a future evolution path of the defect; calculating the gradient change of the spatial field intensity distribution map, combining the future evolution path to perform comprehensive modeling to obtain a dynamic field intensity update sequence; according to the dynamic field intensity update sequence, combining the real-time acquired field intensity distribution data to perform diffusion rate calculation and quality assessment to obtain a final quality detection report.

2. The FPC connector quality detection method based on multi-modal fusion of claim 1, wherein, The method comprises the following steps: performing smoothing processing on the original multi-modal data set by using a mean filtering method to obtain a denoised data set; performing timestamp synchronization processing on the signals of different modalities in the denoised data set, and filling in the missing values in the synchronization process by using a linear interpolation method to obtain a time series synchronized data set; performing normalization processing on the time series synchronized data set to map the data of different physical dimensions to a unified scale to obtain a standardized multi-physical field sequence.

3. The FPC connector quality detection method based on multi-modal fusion of claim 1, wherein, The method comprises the following steps: processing the standardized multi-physical field sequence by using a convolutional neural network, extracting multi-dimensional spatial features, and generating a preliminary field intensity distribution map; calculating the field intensity deviation value of the preliminary field intensity distribution map, and optimizing the parameters of the convolutional neural network according to the field intensity deviation value by using a gradient descent method to obtain an optimized field intensity distribution map; performing clustering grouping on the optimized field intensity distribution map, calculating the spatial correlation between the grouped regions, and performing dimension reduction processing on the spatial correlation by using a principal component analysis method to obtain a reduced feature set; storing the reduced feature set into a pre-established distributed database, and organizing the reduced feature set by using a time series index to obtain a spatial field intensity distribution map.

4. The FPC connector quality detection method based on multi-modal fusion of claim 1, wherein, The method comprises the following steps: performing median filtering preprocessing on the spatial field intensity distribution map, and judging whether the preprocessed data value exceeds a preset field intensity threshold, and determining the region exceeding the threshold as an abnormal region; dividing the abnormal region by using density clustering, grouping the regions with high field intensity values and dense space to obtain a high-density scene region set; Extract feature vectors of each region in the high-density scene region set, and calculate the spatial correlation between each region by using a cosine similarity method to obtain a correlation matrix; Perform dimensionality reduction processing on the multi-dimensional features in the correlation matrix by using a linear discriminant analysis method to obtain a potential defect region list.

5. The FPC connector quality detection method based on multi-modal fusion of claim 1, wherein, The spatial data of the potential defect region list is obtained, a connector internal three-dimensional structure model is constructed, structure analysis and abnormal point positioning are performed, and accurate defect position coordinates are obtained, including: According to the potential defect region list, stereoscopic microscopic three-dimensional scanning is performed on the target region to generate three-dimensional point cloud data; Perform data fusion processing on the three-dimensional point cloud data, integrate multi-modal data such as field strength distribution and temperature signal, and construct a preliminary three-dimensional structure model of the connector internal; Perform structure analysis and feature extraction on the preliminary three-dimensional structure model, identify key structural features, and determine the geometric abnormal points therein; Perform coordinate calculation and correction on the geometric abnormal points to obtain accurate defect position coordinates.

6. The FPC connector quality detection method based on multi-modal fusion of claim 1, wherein, The multi-time observation values of the accurate defect position coordinates are obtained, time series evolution mode learning and future trend prediction are performed on the observation values, and the future evolution path of the defect is obtained, including: Timestamps are sorted to obtain an ordered sequence of observation values, the difference values of adjacent observation values in the ordered sequence are calculated, and all difference values exceeding a preset change threshold are collected to form a defect change increment sequence; Input the defect change increment sequence into a time series model for layer-by-layer propagation and learning to obtain a state hidden representation representing the overall evolution mode of the sequence; According to the state hidden representation, the trend continuation direction is judged and extrapolated to obtain the future evolution path of the defect.

7. The FPC connector quality detection method based on multi-modal fusion of claim 1, wherein, The gradient change of the spatial field strength distribution map is calculated, and the future evolution path is combined for comprehensive modeling to obtain a dynamic field strength update sequence, including: Perform gradient calculation on the spatial field strength distribution map to extract the change rate of the field strength in space to obtain a gradient value sequence; Determine whether the difference between adjacent values in the gradient value sequence exceeds a preset gradient stability threshold, and if the threshold is exceeded, fuse multiple possible branches of the future evolution path to obtain a fused path branch sequence; According to the fused path branch sequence, the thermal stress concentration area of the uneven region is located, and the fluctuation sequence of the real-time environmental variable is determined; Data fusion is performed on the gradient value sequence, the position data of the thermal stress concentration area, and the fluctuation sequence of the environmental variable to finally generate the dynamic field strength update sequence.

8. The FPC connector quality detection method based on multi-modal fusion of claim 1, wherein, According to the dynamic field strength update sequence, the field strength distribution data obtained in real time is combined for diffusion rate calculation and quality evaluation to obtain a final quality detection report, including: Apply the dynamic field strength update sequence to the field strength distribution data obtained in real time to generate predictive enhanced field strength distribution data; Perform spatial feature extraction on the predictive enhanced field strength distribution data, and calculate the change of the features over time to obtain a defect diffusion rate vector; According to the defect diffusion rate vector, a three-dimensional model is constructed, and environmental fluctuations and thermal stress data are fused to generate an enhanced three-dimensional coordinate sequence; The enhanced three-dimensional coordinate sequence is compared with a defect diffusion rate vector and a preset quality standard, a specific value of a geometric size and a diffusion rate exceeding a standard limit is calculated, and finally a quality detection report is generated.

9. The FPC connector quality detection system based on multi-modal fusion, characterized in that, Comprise: A multi-modal data acquisition module for acquiring an original multi-modal data set inside the connector, the original multi-modal data set comprising electrical stress, thermal stress and mechanical stress data; A data preprocessing module for mean filtering and time series alignment on the original multi-modal data set to obtain a standardized multi-physical field sequence; A field strength map construction module for spatial feature extraction and time series construction on the standardized multi-physical field sequence to obtain a spatial field strength distribution map; A defect detection module for density clustering and spatial correlation analysis if the field strength distribution in the spatial field strength distribution map exceeds a preset field strength threshold to obtain a potential defect area list; A three-dimensional defect positioning module for obtaining spatial data of the potential defect area list, constructing a three-dimensional structure model inside the connector, performing structure analysis and abnormal point positioning, and obtaining accurate defect position coordinates; An evolutionary path prediction module for obtaining multi-time observation values of the accurate defect position coordinates, performing time series evolution mode learning and future trend prediction on the observation values, and obtaining a future evolutionary path of the defect; A dynamic field strength updating module for calculating the gradient change of the spatial field strength distribution map, combining the future evolutionary path for comprehensive modeling, and obtaining a dynamic field strength updating sequence; A quality evaluation and report module for calculating the diffusion rate and evaluating the quality according to the dynamic field strength updating sequence and combining the real-time acquired field strength distribution data to obtain a final quality detection report.

Images