A touch display screen integration test method
By synchronously collecting and analyzing the electrical, optical, and thermal data of the touch screen, dynamically triggering high-resolution monitoring, and constructing cross-modal feature vectors, the problem of ambiguous defect localization in existing technologies is solved, and efficient and intelligent defect identification and prediction are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BENGBU JIANGFAN TECHNOLOGY CO LTD
- Filing Date
- 2026-04-01
- Publication Date
- 2026-07-10
AI Technical Summary
Existing testing methods for touch displays are difficult to achieve coordinated monitoring of electrical, optical, and thermal properties, and lack the fusion of correlation information from multimodal and multi-scale data. This results in ambiguous defect localization and low accuracy in defect type identification, making it difficult to meet the high-efficiency testing requirements in large-scale mass production scenarios.
The system simultaneously acquires electrical response signals, optical images, and temperature field data from the touch screen. Abnormal events are identified through electrical derivative sequences, and high-resolution optical imaging and temperature monitoring are dynamically triggered. Cross-modal correlation feature vectors are constructed and input into a defect prediction model for analysis. The system then outputs prediction results of defect types and their probabilities of occurrence.
It significantly improves the accuracy and efficiency of defect location and testing, enhances the reliability and intelligence of defect identification, and can accurately distinguish different types of defects and output quantified occurrence probabilities to guide production line quality grading and process optimization.
Smart Images
Figure CN122364010A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to an integrated testing method for touch displays. Background Technology
[0002] In the current field of touch display testing, traditional testing methods mostly adopt a single-modal data acquisition and independent analysis mode, which makes it difficult to achieve coordinated monitoring of electrical, optical, and thermal characteristics. Most solutions rely solely on electrical signals to detect anomalies, lacking visualization of the abnormal area and verification of its thermal state. This leads to vague defect localization and an inability to accurately correlate the root cause of the anomaly with its apparent features. Furthermore, existing technologies generally employ global high-resolution data acquisition, which not only results in large data processing volumes and low testing efficiency but is also prone to misjudgment of defects due to interference from data in irrelevant areas. This makes it difficult to meet the high-efficiency testing requirements of large-scale mass production scenarios, and a precise testing mode based on dynamic triggering of multimodal high-resolution monitoring of electrical anomalies has not yet been realized.
[0003] Furthermore, existing defect identification methods mostly rely on single-dimensional feature analysis and fail to effectively integrate the correlation information of multimodal and multi-scale data. Traditional solutions often use electrical response data or optical image features in isolation for defect judgment, ignoring the evolution of abnormal events in the time dimension and the inherent correlation of cross-modal features. This results in low accuracy of defect type identification and insufficient reliability of probability prediction. Due to the lack of technical means for cross-modal temporal feature fusion and intelligent defect prediction, it is impossible to fully characterize the multi-physics response changes caused by defects, making it difficult to achieve accurate classification and risk assessment of complex defects. This seriously restricts the intelligence and reliability level of touch display testing. Therefore, how to improve the intelligence and reliability of touch display testing has become an urgent problem to be solved. Summary of the Invention
[0004] This invention provides an integrated testing method for touch displays to solve the problems mentioned in the background art.
[0005] To achieve the above objectives, the present invention provides an integrated testing method for touch displays, comprising:
[0006] S1, synchronously acquires the response signal of the touch screen under electrical excitation, the optical image of the screen and the temperature field data, and preprocesses and aligns the acquired data with time to generate time-synchronized electrical timing data, optical image sequence and temperature field sequence.
[0007] S2, perform electrical response anomaly event detection on the electrical timing data to determine the screen area coordinates corresponding to each anomaly event;
[0008] S3, when an abnormal event is detected, high-resolution optical imaging and high-resolution temperature monitoring are dynamically triggered for the coordinates of the screen area to extract the fine optical and thermal features of the area.
[0009] S4. For each abnormal event, based on its corresponding refined optical features, refined thermal features, and features of the corresponding time window in the electrical time series data, construct a cross-modal correlation feature vector, and combine the correlation feature vectors of all events in chronological order into a cross-modal time series feature matrix.
[0010] S5, input the cross-modal temporal feature matrix into the pre-trained defect prediction model for analysis, and output the prediction results of the defect type and occurrence probability of the touch screen.
[0011] In a preferred embodiment, the synchronous acquisition of the touch screen's response signal under electrical excitation, the screen's optical image, and temperature field data, and the preprocessing and time alignment of the acquired data to generate time-synchronized electrical timing data, optical image sequences, and temperature field sequences, includes:
[0012] Based on a preset gate voltage scanning signal, an operation is applied to the touch screen through a programmable power supply, and the source and drain current response curves are acquired simultaneously to obtain an electrical timing data sequence.
[0013] Based on a wide-angle low-resolution optical sensor, optical image acquisition operations are performed on the touch screen to obtain a low-resolution optical image sequence;
[0014] Based on a low-resolution infrared thermal imager, temperature field data acquisition is performed on the touch screen to obtain a low-resolution temperature field sequence.
[0015] The electrical time-series data sequence, the low-resolution optical image sequence, and the low-resolution temperature field sequence are denoised to obtain denoised electrical data, denoised optical image sequence, and denoised temperature field sequence.
[0016] The denoised electrical data, the denoised optical image sequence, and the denoised temperature field sequence are normalized to obtain normalized electrical data, normalized optical image sequence, and normalized temperature field sequence.
[0017] The normalized electrical data, the normalized optical image sequence, and the normalized temperature field sequence are time-stamp aligned to obtain time-synchronized electrical timing data, optical image sequence, and temperature field sequence.
[0018] In a preferred embodiment, the step of performing a timestamp alignment operation on the normalized electrical data, the normalized optical image sequence, and the normalized temperature field sequence to obtain time-synchronized electrical time-series data, optical image sequence, and temperature field sequence includes:
[0019] Interpolation synchronization is performed based on data timestamps to generate time-consistent data sequences. The specific linear interpolation algorithm used is as follows:
[0020] ;
[0021] In the formula, y t For the interpolation at time t after alignment, y t1 and y t2 These are the original values for adjacent time points t1 and t2, where t1 and t2 are two adjacent time points.
[0022] In a preferred embodiment, the step of detecting electrical response anomalies in the electrical timing data to determine the screen region coordinates corresponding to each anomaly includes:
[0023] Based on the preprocessed electrical time-series data, the first and second derivatives of the current response are calculated to obtain the electrical derivative sequence.
[0024] Based on a preset derivative threshold, an anomaly detection operation is performed on the electrical derivative sequence to obtain an initial set of anomalies.
[0025] A voltage-coordinate mapping operation is performed on the initial set of abnormal points and the preset gate voltage scan signal to obtain the screen area coordinates corresponding to each abnormal event.
[0026] In a preferred embodiment, the step of dynamically triggering high-resolution optical imaging and high-resolution temperature monitoring for the screen area coordinates when an abnormal event is detected, in order to extract refined optical and thermal features of the area, includes:
[0027] Based on the screen area coordinates corresponding to each abnormal event, a dynamic trigger command is generated.
[0028] Based on the dynamic triggering command, the high-resolution optical imaging device is controlled to perform a local rapid shooting operation on the screen area coordinates, and at the same time, the high-resolution temperature monitoring device is controlled to perform a dense temperature measurement operation on the screen area coordinates to obtain a high-resolution optical sub-image and a high-resolution temperature field sub-map.
[0029] Based on the high-resolution optical sub-image, local contrast features and texture features are extracted. At the same time, based on the high-resolution temperature field sub-image, the highest temperature features and temperature gradient features are extracted to obtain the refined optical features and refined thermal features of the region.
[0030] In a preferred embodiment, the control of the high-resolution optical imaging device to perform a local rapid imaging operation on the screen area coordinates, and the control of the high-resolution temperature monitoring device to perform a dense temperature measurement operation on the screen area coordinates, includes:
[0031] Based on the coordinates in the dynamic trigger command, the high-resolution optical imaging device is controlled to perform a local rapid imaging operation to obtain a high-resolution optical sub-image.
[0032] Based on the coordinates in the dynamic trigger command, the high-resolution temperature monitoring device is controlled to perform intensive temperature measurement operations to obtain a high-resolution temperature field sub-map.
[0033] In a preferred embodiment, the step of constructing a cross-modal correlation feature vector for each anomalous event, based on its corresponding refined optical features, refined thermal features, and features of the corresponding time window in the electrical time-series data, includes:
[0034] Based on the occurrence time of each abnormal event, data corresponding to the time window is extracted from the time-synchronized electrical timing data to obtain electrical time window data;
[0035] Based on the electrical time window data and the screen area coordinates corresponding to each abnormal event, statistical features of the region are extracted from the time-synchronized optical image sequence and temperature field sequence to obtain global optical statistical features and global thermal statistical features.
[0036] The refined optical features, the refined thermal features, the electrical features extracted from the electrical time window data, the global optical statistical features, and the global thermal statistical features are concatenated to obtain a cross-modal correlation feature vector for the anomalous event.
[0037] In a preferred embodiment, the step of extracting data corresponding to a time window from the time-synchronized electrical timing data to obtain electrical time window data includes:
[0038] From the time-synchronized electrical timing data, a continuous segment of timing data is selected, centered on the time of the abnormal event, and extended before and after by a preset time length.
[0039] In a preferred embodiment, combining the associated feature vectors of all events in chronological order into a cross-modal temporal feature matrix includes:
[0040] Arrange the cross-modal correlation feature vectors of all events in chronological order to form a cross-modal temporal feature matrix.
[0041] In a preferred embodiment, the step of inputting the cross-modal temporal feature matrix into a pre-trained defect prediction model for analysis, and outputting prediction results regarding the defect type and probability of occurrence of the touch display screen, includes:
[0042] The cross-modal temporal feature matrix is input into a pre-trained defect prediction model to obtain the model output vector;
[0043] The model output vector is subjected to a classification probability calculation operation to obtain the defect type probability distribution;
[0044] The probability distribution of the defect types is subjected to threshold determination and sorting operations to output the predicted results of the defect types and occurrence probabilities of the touch screen.
[0045] Compared with the prior art, the present invention has the following beneficial effects:
[0046] 1. This invention significantly improves the accuracy and efficiency of testing through an innovative design based on dynamic triggering of multimodal high-resolution monitoring of electrical anomalies. The method first synchronously collects global data of electrical, optical, and temperature fields at low resolution. After accurately identifying abnormal events and locating regional coordinates through electrical derivative sequences, high-resolution optical imaging and intensive temperature measurement are dynamically triggered only for suspected areas. This avoids the massive data redundancy caused by global high-resolution acquisition, greatly reducing the data processing load and testing time. At the same time, local high-resolution data acquisition can capture microscopic defect features that are easily missed by traditional global monitoring, such as fine scratches and micro short circuits. Combined with refined feature extraction such as local contrast and temperature gradient, it realizes the leap from defect "discovery" to "precise characterization", effectively improving the accuracy of defect location and the comprehensiveness of feature description.
[0047] 2. This invention relies on cross-modal temporal feature fusion and intelligent defect prediction technology, which greatly enhances the reliability and intelligence level of defect identification. By integrating electrical time window features, refined optical and thermal features, and global statistical features to construct cross-modal correlation feature vectors, and combining them in chronological order into a temporal feature matrix, it comprehensively depicts the correlation and temporal evolution of multi-physics response caused by defects. This breaks through the limitations of traditional single-modal feature analysis. With the help of a pre-trained machine learning model, the fused features are deeply mined, and the probability quantification and ranking of defect types are achieved by combining the Softmax function. It can not only accurately distinguish different types of defects, but also output the quantified probability of occurrence, providing a scientific basis for decision-making. At the same time, the preset probability threshold effectively balances the risks of false alarms and false negatives, making the test results both rigorous and practical. It can directly guide production line quality grading, maintenance decisions, and process optimization, significantly improving the intelligent and industrial application value of touch screen testing. Attached Figure Description
[0048] Figure 1 This is a flowchart illustrating an integrated testing method for a touch display screen according to an embodiment of the present invention.
[0049] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0050] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0051] This application provides an integrated testing method for touch displays. The executing entity of this integrated testing method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the integrated testing method for touch displays can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0052] Reference Figure 1 The diagram shown is a flowchart illustrating an integrated touchscreen testing method according to an embodiment of the present invention. In this embodiment, the integrated touchscreen testing method includes:
[0053] S1, synchronously acquires the response signal of the touch screen under electrical excitation, the optical image of the screen and the temperature field data, and preprocesses and aligns the acquired data with time to generate time-synchronized electrical timing data, optical image sequence and temperature field sequence.
[0054] In this embodiment of the invention, the synchronous acquisition of the response signal of the touch screen under electrical excitation, the optical image of the screen, and the temperature field data, and the preprocessing and time alignment of the acquired data to generate time-synchronized electrical timing data, optical image sequences, and temperature field sequences, includes:
[0055] Based on a preset gate voltage scanning signal, an operation is applied to the touch screen through a programmable power supply, and the source and drain current response curves are acquired simultaneously to obtain an electrical timing data sequence.
[0056] Based on a wide-angle low-resolution optical sensor, optical image acquisition operations are performed on the touch screen to obtain a low-resolution optical image sequence;
[0057] Based on a low-resolution infrared thermal imager, temperature field data acquisition is performed on the touch screen to obtain a low-resolution temperature field sequence.
[0058] The electrical time-series data sequence, the low-resolution optical image sequence, and the low-resolution temperature field sequence are denoised to obtain denoised electrical data, denoised optical image sequence, and denoised temperature field sequence.
[0059] The denoised electrical data, the denoised optical image sequence, and the denoised temperature field sequence are normalized to obtain normalized electrical data, normalized optical image sequence, and normalized temperature field sequence.
[0060] The normalized electrical data, the normalized optical image sequence, and the normalized temperature field sequence are time-stamp aligned to obtain time-synchronized electrical timing data, optical image sequence, and temperature field sequence.
[0061] It should be noted that the operation involves applying a voltage signal to the touch screen under simulated actual working conditions to induce changes in the electrical signal, and combining this with synchronous acquisition to ensure consistent data initiation over time.
[0062] It should be noted that the electrical timing data sequence is an array of current values in a time series, representing the response amplitude of the screen to electrical excitation at different points in time, which is used for subsequent detection of electrical abnormal events.
[0063] It should be noted that the data acquisition operation uses a wide-angle sensor to capture full-screen visual information, continuously monitoring changes in the screen surface at low resolution to ensure data coverage of the entire screen area.
[0064] It should be noted that low-resolution optical image sequences are time-series images that provide screen optical appearance features and serve as a benchmark for dynamically triggered high-resolution imaging.
[0065] It should be noted that temperature field data is acquired by monitoring the heat distribution on the screen surface using an infrared thermal imager. The low-resolution design reduces the data processing load while capturing global temperature change trends.
[0066] It should be noted that the low-resolution temperature field sequence is a time-series thermal value matrix that reflects the thermal characteristics of the screen and is used to identify abnormal heat accumulation.
[0067] It should be noted that the noise reduction operation removes high-frequency noise components from the original data, which is achieved through a median filtering algorithm to improve the signal-to-noise ratio of the data.
[0068] Furthermore, the execution process of the median filtering algorithm is as follows: select 5 time point data points as a fixed window size, sort the data within the window, and replace the center point value with the median value to eliminate random noise.
[0069] It should be noted that the normalization operation unifies the scale of data from different modalities. The min-max normalization algorithm is used to perform normalization processing. For each data point, the difference between the data point and the minimum value of the entire sequence is calculated first, and then the difference between the maximum value and the minimum value is calculated. The two differences are divided to make the data point fall into the interval between 0 and 1.
[0070] In this embodiment of the invention, the step of performing a timestamp alignment operation on the normalized electrical data, the normalized optical image sequence, and the normalized temperature field sequence to obtain time-synchronized electrical time-series data, optical image sequence, and temperature field sequence includes:
[0071] Interpolation synchronization is performed based on data timestamps to generate time-consistent data sequences. The specific linear interpolation algorithm used is as follows:
[0072] ;
[0073] In the formula, y t For the interpolation at time t after alignment, y t1 and y t2 These are the original values for adjacent time points t1 and t2, where t1 and t2 are two adjacent time points.
[0074] S2, perform electrical response anomaly event detection on the electrical timing data to determine the screen area coordinates corresponding to each anomaly event;
[0075] In this embodiment of the invention, the step of detecting electrical response anomalies in the electrical timing data to determine the screen area coordinates corresponding to each anomaly includes:
[0076] Based on the preprocessed electrical time-series data, the first and second derivatives of the current response are calculated to obtain the electrical derivative sequence.
[0077] Based on a preset derivative threshold, an anomaly detection operation is performed on the electrical derivative sequence to obtain an initial set of anomalies.
[0078] A voltage-coordinate mapping operation is performed on the initial set of abnormal points and the preset gate voltage scan signal to obtain the screen area coordinates corresponding to each abnormal event.
[0079] It should be noted that the calculation of the first and second derivatives is based on the preprocessed electrical time-series data. The rate of change of current with time, i.e., the first derivative, and the rate of change of the rate of change, i.e., the second derivative, are calculated in real time using a numerical difference algorithm. The aim is to keenly capture abrupt signals through the high-order dynamic characteristics of the current response.
[0080] Furthermore, the mathematical expression for the numerical difference algorithm is as follows:
[0081]
[0082] In the formula, The first derivative, For the second derivative, t i For the sampling time point, I(t) i ) represents the current value at that moment, and Δt represents the sampling time interval.
[0083] It should be noted that the electrical derivative sequence is a time-series data array composed of the values of the first and second derivatives. An anomaly in the first derivative indicates a rapid change in current, while an anomaly in the second derivative indicates a sudden reversal in the trend of change. Together, they constitute a sensitive indicator for anomaly detection.
[0084] It should be noted that the outlier detection operation compares each derivative value in the electrical derivative sequence with a preset derivative threshold. When the absolute value of any derivative value exceeds its corresponding threshold, that time point is determined to be a potential outlier.
[0085] Furthermore, the first derivative threshold is set to 0.15 A / s, and the second derivative threshold is set to 0.25 A / s². It should be noted that the voltage-coordinate mapping operation is based on the timestamp corresponding to each anomaly in the initial anomaly set. It backtracks to query the gate voltage scan signal applied at that moment, and obtains the two-dimensional coordinates of the anomaly on the screen according to the preset correspondence between the gate scan voltage and the physical pixel position on the screen.
[0086] Furthermore, the gate voltage scan signal is a timing voltage signal that drives the row electrodes of the touch display to scan row by row. Each row scan corresponds to a specific physical area on the screen. Therefore, at the moment an anomaly occurs, the gate row currently being voltaged and the column information of the source and drain electrodes can be uniquely mapped to a specific pixel area or sub-region on the screen.
[0087] It should be noted that the screen area coordinates corresponding to each abnormal event are positional information composed of row and column coordinates. This information is used to accurately locate the suspected location of the electrical response abnormality on the touch display panel, providing a precise spatial positioning target for subsequent dynamic triggering of high-resolution optical and temperature monitoring.
[0088] S3, when an abnormal event is detected, high-resolution optical imaging and high-resolution temperature monitoring are dynamically triggered for the coordinates of the screen area to extract the fine optical and thermal features of the area.
[0089] In this embodiment of the invention, the step of dynamically triggering high-resolution optical imaging and high-resolution temperature monitoring for the screen area coordinates when an abnormal event is detected, in order to extract refined optical and thermal features of the area, includes:
[0090] Based on the screen area coordinates corresponding to each abnormal event, a dynamic trigger command is generated.
[0091] Based on the dynamic triggering command, the high-resolution optical imaging device is controlled to perform a local rapid shooting operation on the screen area coordinates, and at the same time, the high-resolution temperature monitoring device is controlled to perform a dense temperature measurement operation on the screen area coordinates to obtain a high-resolution optical sub-image and a high-resolution temperature field sub-map.
[0092] Based on the high-resolution optical sub-image, local contrast features and texture features are extracted. At the same time, based on the high-resolution temperature field sub-image, the highest temperature features and temperature gradient features are extracted to obtain the refined optical features and refined thermal features of the region.
[0093] It should be noted that the process of generating dynamic trigger commands is based on the occurrence time and screen area coordinates of each abnormal event. It automatically generates control commands containing precise timestamps and spatial coordinates, aiming to immediately and accurately start the high-resolution data acquisition process for the suspected abnormal area, and realize on-demand, fixed-point data capture driven by electrical abnormal events.
[0094] Furthermore, the essence of this instruction is a structured control signal packet that transforms the abstract "time-abnormal intensity" information of electrical detection into "when-where" acquisition tasks that can be performed by optical and thermal sensors, thereby establishing precise coordination of cross-modal sensing.
[0095] It should be noted that extracting local contrast features involves calculating the maximum and minimum grayscale difference within a specific neighborhood of the high-resolution optical sub-image, aiming to quantify the degree of brightness variation in that area. Certain defects can cause significant optical contrast differences between the edges and the background, and this feature is a key optical indicator for identifying such defects.
[0096] It should be noted that the extraction of texture features is performed on the high-resolution optical sub-image by extracting numerical vectors that describe the roughness, regularity, and directionality of the texture of the region.
[0097] It should be noted that extracting the highest temperature feature involves finding the maximum value among all temperature measurement points in the high-resolution temperature field submap, directly quantifying the peak temperature of the "hot spots" that may exist in the abnormal area, in order to obtain important thermal evidence of serious defects such as short circuits and overcurrents.
[0098] It should be noted that the extraction of temperature gradient features is based on the high-resolution temperature field submap and the calculation of its spatial temperature change rate. This is usually obtained by calculating the temperature difference between adjacent temperature measurement points. The temperature gradient features represent the diffusion of heat in the abnormal region. A steep temperature gradient may indicate that the heat source is very localized, while a gentle gradient may indicate that the heat source area is large or the heat dissipation conditions are different. This is of great value in distinguishing defect types.
[0099] It should be noted that the refined optical features are numerical vectors composed of the local contrast features and the texture features, etc. This vector comprehensively and quantitatively describes the microscopic appearance of the abnormal region under the optical mode, providing accurate input for subsequent correlation analysis with electrical and thermal features.
[0100] In this embodiment of the invention, the control of the high-resolution optical imaging device to perform a local rapid imaging operation on the screen area coordinates, and the control of the high-resolution temperature monitoring device to perform a dense temperature measurement operation on the screen area coordinates, includes:
[0101] Based on the coordinates in the dynamic trigger command, the high-resolution optical imaging device is controlled to perform a local rapid imaging operation to obtain a high-resolution optical sub-image.
[0102] Based on the coordinates in the dynamic trigger command, the high-resolution temperature monitoring device is controlled to perform intensive temperature measurement operations to obtain a high-resolution temperature field sub-map.
[0103] It should be noted that controlling the high-resolution optical imaging device to perform local rapid shooting operation involves driving the gimbal or lens of the high-resolution CMOS camera to quickly align with the area according to the coordinates in the dynamic trigger command, and performing rapid imaging of the local area within the time specified in the command.
[0104] Furthermore, the local rapid shooting operation focuses on a small area of the screen corresponding to the abnormal coordinates, such as a pixel block or an area a few millimeters square, rather than full-screen shooting, to capture the microscopic optical properties of the area with high spatial resolution, such as fine scratches, bright spots, dark spots or color unevenness, overcoming the problem of insufficient detail in global low-resolution monitoring.
[0105] It should be noted that controlling the high-resolution temperature monitoring device to perform intensive temperature measurement operations involves driving a high-precision infrared thermocouple array or microbolometer array to perform a high spatial resolution temperature scan of the area based on the coordinates in the dynamic trigger command.
[0106] Furthermore, the intensive temperature measurement operation involves intensive spatial point sampling of the local area at the moment of or shortly after the anomaly occurs. This operation is used to obtain a detailed temperature distribution map of the area, which can detect local hot spots or abnormal temperature gradients with a small spatial range caused by defects such as micro short circuits or abnormal contact resistance.
[0107] It should be noted that the high-resolution optical sub-image is a high-definition image data that only contains the local area corresponding to the screen area coordinates. Its pixel resolution is much higher than that of the low-resolution image during global monitoring. Physically, it provides refined visual information on the surface state of abnormal areas and is a direct data source for the optical characterization of defects.
[0108] It should be noted that the high-resolution temperature field submap is a refined temperature distribution matrix that only contains the local area corresponding to the screen area coordinates, and its spatial density of temperature measurement points is much higher than that of a global low-resolution thermal imager.
[0109] S4. For each abnormal event, based on its corresponding refined optical features, refined thermal features, and features of the corresponding time window in the electrical time series data, construct a cross-modal correlation feature vector, and combine the correlation feature vectors of all events in chronological order into a cross-modal time series feature matrix.
[0110] In this embodiment of the invention, the step of constructing a cross-modal correlation feature vector for each anomalous event, based on its corresponding refined optical features, refined thermal features, and features of the corresponding time window in the electrical time-series data, includes:
[0111] Based on the occurrence time of each abnormal event, data corresponding to the time window is extracted from the time-synchronized electrical timing data to obtain electrical time window data;
[0112] Based on the electrical time window data and the screen area coordinates corresponding to each abnormal event, statistical features of the region are extracted from the time-synchronized optical image sequence and temperature field sequence to obtain global optical statistical features and global thermal statistical features.
[0113] The refined optical features, the refined thermal features, the electrical features extracted from the electrical time window data, the global optical statistical features, and the global thermal statistical features are concatenated to obtain a cross-modal correlation feature vector for the anomalous event.
[0114] It should be noted that the electrical features extracted from the electrical time window data are quantitative indicators calculated based on the data segment, such as the drift of the threshold voltage, the change of the subthreshold swing, or the offset of the mean current within the window.
[0115] Furthermore, the threshold voltage drift is calculated by first extracting the 95th percentile of all mathematically reversed sorted data from the electrical data at the beginning of the time window using a standard method to obtain a threshold voltage value; secondly, extracting the 94th percentile of all mathematically reversed sorted data from the electrical data at the end of the time window using the same method to obtain another threshold voltage value; finally, calculating the difference between the two values to obtain the threshold voltage drift ΔVth.
[0116] It should be noted that the statistical features of the region extracted from the global sequence are based on the screen region coordinates corresponding to each abnormal event. The average value of all temperature measurement points in the region is located and calculated from the time-synchronized low-resolution optical image sequence and low-resolution temperature field sequence within the same time window.
[0117] Furthermore, the purpose of extracting statistical features is to obtain the macroscopic optical and thermal performance of the abnormal region from a global, continuous monitoring perspective, and to quantify its overall brightness level and thermal equilibrium state. These features complement the refined features extracted at high resolution.
[0118] It should be noted that the splicing operation combines feature values from five different sources and data dimensions—electricity, refined optics, refined thermal, global optics, and global thermal—in a predetermined order into a longer, one-dimensional numerical array.
[0119] Furthermore, the physical significance of splicing is to achieve the fusion of multimodal and multi-scale feature information, integrating the root cause of electrical anomalies, the optical appearance changes they cause, thermal effects, and macroscopic manifestations into a unified mathematical representation, providing comprehensive input features for subsequent defect classification models based on machine learning.
[0120] It should be noted that the cross-modal correlation feature vector is a high-dimensional numerical vector that integrates electrical transient characteristics, optical microscopic and macroscopic characteristics, and thermal microscopic and macroscopic characteristics. It is a complete, integrated, and digital characterization of the multi-physics response state corresponding to a single electrical anomaly event. This vector establishes the intrinsic correlation between different modal features under the "same anomaly event" and is the core data foundation for achieving high-precision defect type identification and prediction.
[0121] In this embodiment of the invention, the step of extracting data corresponding to a time window from the time-synchronized electrical timing data to obtain electrical time window data includes:
[0122] From the time-synchronized electrical timing data, a continuous segment of timing data is selected, centered on the time of the abnormal event, and extended before and after by a preset time length.
[0123] It should be noted that in the preset time extensions before and after, the extension is 3 seconds before and 5 seconds after.
[0124] Furthermore, the purpose of capturing the time window is to obtain electrical response data over a period of time before and after the occurrence of the abnormal event, aiming to capture the changing trends or transient processes of electrical parameters related to the defect, rather than just isolated anomalies.
[0125] It should be noted that electrical time window data is a continuous segment of electrical signal data that includes the moment when the abnormal event occurs and the moments before and after it. It reflects the complete process of transient changes in electrical characteristics or continuous deviations from the normal state caused by potential defects, and provides a data foundation for extracting richer electrical features.
[0126] In this embodiment of the invention, the step of combining the associated feature vectors of all events in chronological order into a cross-modal temporal feature matrix includes:
[0127] Arrange the cross-modal correlation feature vectors of all events in chronological order to form a cross-modal temporal feature matrix.
[0128] S5, input the cross-modal temporal feature matrix into the pre-trained defect prediction model for analysis, and output the prediction results of the defect type and occurrence probability of the touch screen.
[0129] In this embodiment of the invention, the step of inputting the cross-modal temporal feature matrix into a pre-trained defect prediction model for analysis, and outputting prediction results regarding the defect type and probability of occurrence of the touch display screen, includes:
[0130] The cross-modal temporal feature matrix is input into a pre-trained defect prediction model to obtain the model output vector;
[0131] The model output vector is subjected to a classification probability calculation operation to obtain the defect type probability distribution;
[0132] The probability distribution of the defect types is subjected to threshold determination and sorting operations to output the predicted results of the defect types and occurrence probabilities of the touch screen.
[0133] It should be noted that inputting the cross-modal temporal feature matrix into the pre-trained defect prediction model is based on the matrix. Through the forward propagation calculation of the model, the high-dimensional temporal features are mapped to the abstract representation space of the defect category. The aim is to use the knowledge learned by the model to comprehensively analyze and transform the input features.
[0134] Furthermore, forward propagation computation is the process of performing mathematical operations such as matrix multiplication and activation functions in a layer-by-layer sequence within the model. Its physical meaning is to extract and combine feature information layer by layer, ultimately forming an abstract representation of the input data.
[0135] It should be noted that the defect prediction model is a machine learning model trained using historical touch screen test data. It learns the complex nonlinear mapping relationship between multimodal temporal features and various defect types, and has the ability to identify defect patterns from fused features.
[0136] It should be noted that the model output vector is the activation value vector of the last hidden layer of the defect prediction model. It is a high-level abstract feature representation obtained after the input cross-modal temporal features are transformed by the model nonlinearly. This vector encodes key information related to defect discrimination and provides a basis for subsequent probability calculation.
[0137] It should be noted that the classification probability calculation operation is based on the model output vector. The original output score is converted into a probability value for each defect category through a normalized exponential function, which aims to transform the abstract output of the model into an intuitive probability form.
[0138] Furthermore, the mathematical expression for the Softmax function is as follows:
[0139]
[0140] In the formula, P(y=j|z) is the conditional probability that the input feature belongs to the j-th type of defect, z is the model output vector, which is a real number array of length K. j Let be the element value in vector z corresponding to the j-th type of defect, and K be the preset total number of defect types.
[0141] It should be noted that the defect type probability distribution is a probability array composed of all defect categories and their corresponding probability values. It quantifies the likelihood that the current touch screen test data belongs to each preset defect category. The higher the probability value, the higher the confidence level of belonging to that type of defect, providing a statistically based quantitative basis for the final decision.
[0142] It should be noted that the threshold determination and sorting operation is based on a preset probability threshold to filter and sort the probability distribution of the defect types, with the aim of extracting the most likely defect types that are significantly higher than the background noise from the probability distribution.
[0143] Furthermore, the preset probability threshold is set to 0.7 to distinguish between "potential defects" and "background fluctuations," control the strictness of the prediction results, and balance false alarms and false negatives.
[0144] It should be noted that the sorting operation arranges the defect categories retained after threshold filtering according to their corresponding probability values from high to low, clarifying the primary and secondary order of multiple potential defects, and placing the most likely defect type first.
[0145] It should be noted that the predicted results of the defect types and occurrence probabilities of the touch display screen are an ordered list. Each item in the list contains a defect type label and its corresponding occurrence probability value, which provides a final, quantitative diagnostic conclusion about the health status of the screen. This conclusion integrates multimodal evidence from electrical, optical, and thermal perspectives and can be directly used to guide maintenance decisions, quality grading, or production line process optimization.
[0146] In the several embodiments provided by this invention, it should be understood that the disclosed method can be implemented in other ways.
[0147] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0148] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, and technology that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.
[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method for testing an integrated touch display screen, characterized in that, The method includes: S1, synchronously acquires the response signal of the touch screen under electrical excitation, the optical image of the screen and the temperature field data, and preprocesses and aligns the acquired data with time to generate time-synchronized electrical timing data, optical image sequence and temperature field sequence. S2, perform electrical response anomaly event detection on the electrical timing data to determine the screen area coordinates corresponding to each anomaly event; S3, when an abnormal event is detected, high-resolution optical imaging and high-resolution temperature monitoring are dynamically triggered for the coordinates of the screen area to extract the fine optical and thermal features of the area. S4. For each abnormal event, based on its corresponding refined optical features, refined thermal features, and features of the corresponding time window in the electrical time series data, construct a cross-modal correlation feature vector, and combine the correlation feature vectors of all events in chronological order into a cross-modal time series feature matrix. S5, input the cross-modal temporal feature matrix into the pre-trained defect prediction model for analysis, and output the prediction results of the defect type and occurrence probability of the touch screen.
2. The integrated testing method for a touch display screen as described in claim 1, characterized in that, The synchronous acquisition of the touch screen's response signal under electrical excitation, the screen's optical image, and temperature field data, and the preprocessing and time alignment of the acquired data to generate time-synchronized electrical timing data, optical image sequences, and temperature field sequences, including: Based on a preset gate voltage scanning signal, an operation is applied to the touch screen through a programmable power supply, and the source and drain current response curves are acquired simultaneously to obtain an electrical timing data sequence. Based on a wide-angle low-resolution optical sensor, optical image acquisition operations are performed on the touch screen to obtain a low-resolution optical image sequence; Based on a low-resolution infrared thermal imager, temperature field data acquisition is performed on the touch screen to obtain a low-resolution temperature field sequence. The electrical time-series data sequence, the low-resolution optical image sequence, and the low-resolution temperature field sequence are denoised to obtain denoised electrical data, denoised optical image sequence, and denoised temperature field sequence. The denoised electrical data, the denoised optical image sequence, and the denoised temperature field sequence are normalized to obtain normalized electrical data, normalized optical image sequence, and normalized temperature field sequence. The normalized electrical data, the normalized optical image sequence, and the normalized temperature field sequence are time-stamp aligned to obtain time-synchronized electrical timing data, optical image sequence, and temperature field sequence.
3. The integrated testing method for a touch display screen as described in claim 2, characterized in that, The step of performing timestamp alignment on the normalized electrical data, the normalized optical image sequence, and the normalized temperature field sequence to obtain time-synchronized electrical time-series data, optical image sequence, and temperature field sequence includes: Interpolation synchronization is performed based on data timestamps to generate time-consistent data sequences. The specific linear interpolation algorithm used is as follows: ； In the formula, y t For the interpolation at time t after alignment, y t1 and y t2 These are the original values for adjacent time points t1 and t2, where t1 and t2 are two adjacent time points.
4. The integrated testing method for a touch display screen as described in claim 1, characterized in that, The step of detecting electrical response anomalies in the electrical timing data to determine the screen region coordinates corresponding to each anomaly includes: Based on the preprocessed electrical timing data, the first and second derivatives of the current response are calculated to obtain the electrical derivative sequence. Based on a preset derivative threshold, an anomaly detection operation is performed on the electrical derivative sequence to obtain an initial set of anomalies. A voltage-coordinate mapping operation is performed on the initial set of abnormal points and the preset gate voltage scan signal to obtain the screen area coordinates corresponding to each abnormal event.
5. The integrated testing method for a touch display screen as described in claim 1, characterized in that, When an abnormal event is detected, high-resolution optical imaging and high-resolution temperature monitoring are dynamically triggered for the screen area coordinates to extract refined optical and thermal features of the area, including: Based on the screen area coordinates corresponding to each abnormal event, a dynamic trigger command is generated. Based on the dynamic triggering command, the high-resolution optical imaging device is controlled to perform a local rapid shooting operation on the screen area coordinates, and at the same time, the high-resolution temperature monitoring device is controlled to perform a dense temperature measurement operation on the screen area coordinates to obtain a high-resolution optical sub-image and a high-resolution temperature field sub-map. Based on the high-resolution optical sub-image, local contrast features and texture features are extracted. At the same time, based on the high-resolution temperature field sub-image, the highest temperature features and temperature gradient features are extracted to obtain the refined optical features and refined thermal features of the region.
6. The integrated testing method for a touch display screen as described in claim 5, characterized in that, The control of the high-resolution optical imaging device to perform a local rapid imaging operation on the screen area coordinates, and the control of the high-resolution temperature monitoring device to perform a dense temperature measurement operation on the screen area coordinates, includes: Based on the coordinates in the dynamic trigger command, the high-resolution optical imaging device is controlled to perform a local rapid imaging operation to obtain a high-resolution optical sub-image. Based on the coordinates in the dynamic trigger command, the high-resolution temperature monitoring device is controlled to perform intensive temperature measurement operations to obtain a high-resolution temperature field sub-map.
7. The integrated testing method for a touch display screen as described in claim 1, characterized in that, For each anomalous event, a cross-modal correlation feature vector is constructed based on its corresponding refined optical features, refined thermal features, and features of the corresponding time window in the electrical time-series data, including: Based on the occurrence time of each abnormal event, data corresponding to the time window is extracted from the time-synchronized electrical timing data to obtain electrical time window data; Based on the electrical time window data and the screen area coordinates corresponding to each abnormal event, statistical features of the region are extracted from the time-synchronized optical image sequence and temperature field sequence to obtain global optical statistical features and global thermal statistical features. The refined optical features, the refined thermal features, the electrical features extracted from the electrical time window data, the global optical statistical features, and the global thermal statistical features are concatenated to obtain a cross-modal correlation feature vector for the anomalous event.
8. The integrated testing method for a touch display screen as described in claim 7, characterized in that, The step of extracting data corresponding to a time window from the time-synchronized electrical timing data to obtain electrical time window data includes: From the time-synchronized electrical timing data, a continuous segment of timing data is selected, centered on the time of the abnormal event, and extended before and after by a preset time length.
9. The integrated testing method for a touch display screen as described in claim 1, characterized in that, The process of combining the associated feature vectors of all events in chronological order into a cross-modal temporal feature matrix includes: Arrange the cross-modal correlation feature vectors of all events in chronological order to form a cross-modal temporal feature matrix.
10. The integrated testing method for a touch display screen as described in claim 1, characterized in that, The step of inputting the cross-modal temporal feature matrix into a pre-trained defect prediction model for analysis, and outputting prediction results regarding the defect type and probability of occurrence of the touch display screen, includes: The cross-modal temporal feature matrix is input into a pre-trained defect prediction model to obtain the model output vector; The model output vector is subjected to a classification probability calculation operation to obtain the defect type probability distribution; The probability distribution of the defect types is subjected to threshold determination and sorting operations to output the predicted results of the defect types and occurrence probabilities of the touch screen.