A method and system for judging display failure of an LCD display screen

Abstract

The application discloses a kind of LCD display screen display fault judgment methods, it is related to LCD display screen detection relevant technical field, including multi-source signal synchronous acquisition and preprocessing, feature extraction, fault judgment model establishment, dynamic test mode generation, according to preliminary detection result dynamic generation targeted test image excitation potential fault, Q-learning algorithm iteration optimization test sequence, fault diagnosis.The application also discloses a kind of LCD display screen display fault judgment system, including multi-source data acquisition module, central processing unit, memory and user interface.The application is synchronous acquisition multiple data by optical sensor, electric signal probe, thermal imaging module and environmental sensor, including display data, driving voltage / current monitoring data, temperature distribution and environmental parameter etc., can obtain the all-round information of LCD display screen in running process, avoids the limitation and misjudgment possibly brought by single data source.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of LCD display screen detection, and particularly relates to an LCD display screen display fault judgment method and system. BACKGROUND

[0002] With the rapid development of modern technology, liquid crystal display (LCD) as an important interface of human-computer interaction has been widely used in various electronic devices, such as smart phones, tablet computers, televisions, computer monitors, etc. However, due to long-term use, environmental factors and manufacturing process, etc., LCD display screens may have various faults, common ones including pixel damage, display abnormalities, color distortion, response time delay, etc. These faults not only affect the user's visual experience, but also may cause the device to malfunction, and even in some critical scenarios, such as medical, aviation, etc., display screen failure may also cause serious safety problems.

[0003] Currently, the traditional LCD display screen fault judgment method mainly relies on manual inspection and simple test images or screen display observation to determine whether there is a problem. This method has many shortcomings: on the one hand, manual inspection is easily affected by personal subjective judgment, different inspectors may draw different conclusions, leading to misjudgment or missing problems; on the other hand, some existing technologies only use a single detection method, such as only using image comparison analysis to judge whether the screen has a fault, which is difficult to accurately detect and diagnose for some complex fault types or early potential faults. For example, relying only on appearance inspection or simple electrical signal testing may not be able to find deep-seated problems such as internal circuit aging and abnormal liquid crystal molecule arrangement.

[0004] Therefore, it is necessary to propose an LCD display screen display fault judgment method and system to solve the above problems. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides an LCD display screen display fault judgment method and system, which solves the problems of the prior art that different inspectors may draw different conclusions, leading to misjudgment or missing problems, and that only using image comparison analysis to judge whether the screen has a fault, which is difficult to accurately detect and diagnose for some complex fault types or early potential faults.

[0006] To achieve the above purpose, the present application is realized by the following technical scheme:

[0007] An LCD display screen display fault judgment method, comprising the following steps:

[0008] Step one: multi-source signal synchronous acquisition and preprocessing: using optical sensors, electrical signal probes, thermal imaging modules and environmental sensors to synchronously collect display data, driving voltage / current monitoring data, temperature distribution and environmental parameters of the LCD display screen, and pre-processing the collected data, including denoising, adjusting contrast and brightness to ensure the quality and consistency of the images, and storing the data;

[0009] Step two: feature extraction: extracting key features from the pre-processed images, including pixels, texture features, analyzing the changes of electrical signals, temperature distribution and environmental parameters, and extracting feature information related to faults;

[0010] Step three: fault judgment model establishment: building a multi-dimensional feature fault library containing several fault cases, associating fault phenomena with underlying causes (such as TFT electric leakage, COF virtual welding);

[0011] Step four: dynamic test mode generation: dynamically generating targeted test images according to the preliminary detection results to stimulate potential faults; reinforcement learning optimization path: iteratively optimizing the test sequence through Q-learning algorithm to shorten the detection time while ensuring coverage;

[0012] Step five: fault diagnosis: collecting the detection results of the optimized test image sequence on the display screen, comprehensively evaluating whether the display screen has faults and the type and severity of the faults, and obtaining the fault diagnosis report of the display screen.

[0013] Optionally, the multi-source signal synchronous acquisition method of step one is:

[0014] S1: connect the optical sensor to the data acquisition host through the data line, the installation position of the optical sensor covers the visible area of the display screen, and avoids occlusion and reflection, connect the electrical signal probe to the node of the driving circuit of the display screen to ensure good electrical contact, connect the probe to the data acquisition host through a shielded cable to reduce external electromagnetic interference; adjust the focal length and position of the thermal imaging module to focus on the surface of the display screen, transmit the thermal imaging data to the data acquisition host through the Ethernet interface, install the environmental sensor in the area where the display is located, and communicate with the data acquisition host through the interface;

[0015] S2: create a main thread to coordinate the data acquisition tasks of each sensor, and create independent sub-threads for each sensor;

[0016] S3: For the optical sensor thread, when the main thread receives the start acquisition instruction, the sub-thread starts to obtain image data from the camera at the set frame rate. In the electrical signal probe sub-thread, the voltage and current data transmitted by the probe are read in a loop and stored in the pre-set buffer. The thermal imaging module sub-thread obtains thermal imaging data at the set time interval. The environmental sensor sub-thread continuously acquires environmental light and humidity data and updates the corresponding variables.

[0017] S4: Define the storage format for each type of sensor data.

[0018] Optionally, the feature extraction method in step two is:

[0019] S1: Optical feature extraction: extract pixel statistical features, calculate statistical parameters such as mean, variance, and standard deviation of the image;

[0020] S2: Electrical signal feature extraction: extract time domain features, calculate the effective value, average value, and peak value of the voltage and current signals;

[0021] S3: Thermal imaging feature extraction: extract temperature statistical features, calculate the average temperature, maximum temperature, and minimum temperature of the thermal imaging image, and analyze the features of the temperature histogram;

[0022] S4: Obtain the light intensity value measured by the ambient light sensor and the humidity value measured by the humidity sensor as environmental features.

[0023] Optionally, the construction method of the multi-dimensional feature fault library in step three is: integrate the data collected from each sensor and extracted features to form a data set containing multiple feature vectors. For each set of collected data, combine the corresponding image feature vector, electrical signal feature vector, thermal imaging feature vector, and environmental feature vector. Label the collected data, including whether there is a fault and the type of fault. Select a database management system to store the multi-dimensional fault feature library, design the database table structure, including the data table, fault type table, and sensor information table. The data table is used to store the collected data and the corresponding feature vector. The fault type table records the information of different fault types. The sensor information table stores the relevant parameters and state information of the sensor.

[0024] Optionally, the multi-source signal synchronous acquisition and preprocessing in step one includes a time sequence association module, which includes a hardware clock synchronization unit, a software timestamp calibration unit, and an association rule mining unit.

[0025] Hardware clock synchronization unit: a hardware clock synchronization device is used in the multi-source data acquisition module, and the device clocks of the sensors are synchronized with a standard time server through an NTP protocol; a timer interrupt program is set in the multi-source data acquisition module, and a synchronization signal is sent to each sensor when the timer triggers, so that each sensor records data at the same time and stamps the time;

[0026] Software timestamp calibration unit: in the multi-source data acquisition module, the acquisition time stamp of each collected data sample is recorded, and the data acquisition time stamps of different sensors are calibrated to have the same frequency;

[0027] Correlation rule mining unit: the Apriori algorithm is used to find the correlation rules between sensor data. First, the data set is scanned to find all the frequently occurring single sensor data features, and then the frequently occurring multi-sensor data combination features are found through iteration to finally generate the correlation rules.

[0028] Optionally, the algorithm implementation steps of the Apriori algorithm are:

[0029] S1: Data acquisition and integration: data is collected from optical sensors, electrical signal probes, thermal imaging modules and environmental sensors;

[0030] S2: Transaction database construction: each group of preprocessed data is regarded as a transaction;

[0031] S3: Frequent item set mining: including single sensor data feature frequent item set mining and multi-sensor data combination feature frequent item set mining;

[0032] S4: Association rule generation: for each frequent item set, generate all possible non-empty subset combinations as the antecedent and consequent of the rule, generate the rule, and calculate the confidence of each rule; if the confidence meets the given minimum confidence threshold, the rule is retained;

[0033] S5: Rule optimization: remove redundant rules.

[0034] Optionally, the dynamic test mode generation in step four includes a preliminary detection module, a dynamic test image generation module and a reinforcement learning optimization module;

[0035] Preliminary detection module: the initial image of the display screen is collected by the optical sensor, the image is preprocessed, and then the feature extraction algorithm is used to preliminarily judge the area where the fault may exist;

[0036] Dynamic test image generation module: according to the preliminary detection result, the corresponding test image is generated through the deep neural network, the module contains multiple sub-networks for generating different types of test images, including high-frequency flicker image generation sub-network, gradual gray scale image generation sub-network, polar coordinate grid image generation sub-network, each sub-network receives the output result of the preliminary detection module, and then generates the corresponding test image; reinforcement learning optimization module: the generated test image sequence is optimized by using Q-learning algorithm, the module records the execution effect (evaluated by reward value) of each test image, and adjusts the order and selection of the test sequence according to the records, so as to shorten the detection time while ensuring the coverage rate.

[0037] Optionally, the training process of the high-frequency flicker image generation sub-network is: preparing a plurality of display screen image data with annotations, the annotation content including fault area position and type, taking 70% of the data as the training set and 30% of the data as the verification set; for each training sample, input the annotated fault area information into the neural network, calculate the difference between the output high-frequency flicker image and the actual annotated flicker image, adjust the weight and bias of the neural network through the back propagation algorithm to minimize the difference, verify the performance of the trained neural network on the verification set, and adjust the hyperparameters according to the verification result;

[0038] The training process of the gradual gray scale image generation sub-network is: collecting a plurality of display screen images of different fault types as training data, extracting fault area information from each image and annotating the corresponding gradual gray scale image, training the neural network using a supervised learning method to make the output gradual gray scale image as close as possible to the annotated image, and using a cross-entropy loss function to measure the difference and adjust the model parameters during the training process.

[0039] The training process of the polar coordinate grid image generation sub-network is: constructing a training data set containing fault area information and corresponding polar coordinate grid image annotations, serializing the input data and inputting it into the RNN for training, and evaluating the model performance by calculating the accuracy of the predicted polar coordinate grid image and the annotated image during the training process, and adjusting the model parameters according to the loss value.

[0040] Optionally, the Q-learning algorithm optimizes the generated test image sequence in the following way:

[0041] S1: state definition: the state space S is composed of the following elements: the state information of the current display screen area to be detected, including the results of the detected area and the characteristics of the remaining undetected area; the history record of the selected test image sequence, including the type of the test image, the execution order and the detection result of each test image in this state;

[0042] S2: Action definition: the action space A includes selecting the next test image to be executed, after selecting an action each time, executing the corresponding test image and observing its effect;

[0043] S3: Reward function design: the reward function R is determined according to the change of the state after executing the action, if the test image discovers a new fault area, a positive reward is given, if the test image misjudges the fault area, a negative reward is given;

[0044] S4: Q table update: initialize the Q table, set the Q value of all state-action pairs to 0 or a small random value, update the Q table according to the actual reward after executing the action and the estimated future reward;

[0045] S5: Strategy improvement: select the optimal strategy according to the updated Q table, in each state, select the action with the highest Q value as the optimal action, in the training process, iteratively execute the above steps until the Q table converges or reaches the preset training number of rounds.

[0046] An LCD display screen display fault judgment system, comprising a multi-source data acquisition module, a central processor, a memory and a user interface;

[0047] The multi-source data acquisition module comprises an optical sensor, an electrical signal probe, a thermal imaging module and an environmental sensor and its data acquisition host; the optical sensor is a camera, which is used to acquire image data of the LCD display screen, to obtain the display content, brightness and color information of the screen, so as to analyze whether there is pixel point fault, line fault and color block abnormal display problem; the electrical signal probe is a voltage and current probe, which monitors the driving voltage and current of the LCD display screen, can acquire the waveform, amplitude and frequency parameters of the electrical signal in real time, so as to judge whether the power supply circuit is normal and the signal transmission is stable, which is helpful to find the display abnormality caused by circuit fault; the thermal imaging module is a thermal imager, which detects the temperature distribution of each area of the LCD display screen through thermal imaging technology, and can timely find the fault hidden danger caused by overheating or poor heat dissipation; the environmental sensor is a light sensor and a humidity sensor, which acquires environmental light intensity and humidity environment data;

[0048] The central processor: responsible for coordinating the work of each sensor, receiving the data transmitted by the sensor, and performing preliminary processing and analysis, running the fault judgment algorithm, comparing and judging the data according to the preset rules and models, to determine whether the LCD display screen has fault and the type and severity of the fault;

[0049] Memory: used for storing system programs, fault feature library, sensor calibration parameter data, system programs are software instruction sets for controlling the operation of the entire system; the fault feature library contains the characteristic information of various common faults and the corresponding processing measures; the sensor calibration parameters are used to ensure the accuracy and reliability of sensor measurement;

[0050] User interface: used for setting system parameters, starting and stopping data acquisition and fault judgment process, viewing fault judgment results and historical records, realizing automatic control and operation management of the system, and automatically controlling the working state of each hardware component and the data acquisition process according to the preset conditions and rules.

[0051] The present application provides a LCD display screen display fault judgment method and system, which has the following beneficial effects:

[0052] 1、The present application synchronously collects various data including display data, driving voltage / current monitoring data, temperature distribution and environmental parameters, etc. through optical sensors, electrical signal probes, thermal imaging modules and environmental sensors, which can obtain comprehensive information of the LCD display screen during operation, avoiding the limitations and misjudgments that may be caused by a single data source; the collected data are preprocessed by denoising, adjusting contrast and brightness, etc., which can effectively improve the quality of data, reduce noise interference, make the data more accurate and clear, and be beneficial to subsequent feature extraction and fault judgment.

[0053] 2、The present application constructs a multi-dimensional feature fault library containing several fault cases, and associates the fault phenomena with the underlying causes, which provides valuable reference for fault judgment. In actual fault diagnosis, the type and cause of the fault can be quickly and accurately judged by comparing and analyzing the cases in the fault library, which improves the efficiency and accuracy of fault diagnosis.

[0054] 3、The present application dynamically generates targeted test images according to the preliminary detection results, which can stimulate potential faults and make potential problems more easily exposed. This method of dynamically generating test images can be adjusted and optimized according to actual conditions, which improves the effectiveness and coverage of the test; the test sequence is iteratively optimized by Q-learning algorithm, which shortens the detection time while ensuring the coverage, and the reinforcement learning algorithm can automatically adjust the strategy according to the environmental feedback. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 It is a flowchart of the LCD display screen display fault judgment method of the present application;

[0056] Figure 2 It is a module diagram of the LCD display screen display fault judgment system of the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present application.

[0058] Embodiment 1

[0059] Please refer to Figure 1 A method for judging display failure of an LCD display screen, comprising the following steps:

[0060] Step 1: Synchronous acquisition and preprocessing of multiple source signals: using an optical sensor (high-frame-rate camera), an electrical signal probe, a thermal imaging module and an environmental sensor (such as ambient light, ambient humidity) to synchronously acquire display data, driving voltage / current monitoring data, temperature distribution and environmental parameters of the LCD display screen, and pre-processing the acquired data, including denoising, adjusting contrast and brightness, ensuring the quality and consistency of the images, and storing the data;

[0061] Step 2: Feature extraction: extracting key features from the pre-processed images, including pixel, texture features, analyzing changes in electrical signals, temperature distribution and environmental parameters, and extracting feature information related to failure;

[0062] Step 3: Establishment of failure judgment model: constructing a multi-dimensional feature failure library containing a plurality of failure cases, and associating failure phenomena with underlying causes (such as TFT electric leakage, COF virtual welding);

[0063] Step 4: Dynamic test pattern generation: dynamically generating targeted test images (such as high-frequency flicker, gradual gray scale, polar coordinate grid) according to the preliminary detection results, to stimulate potential failures (such as liquid crystal response delay, pixel sticking), for example, for the area where liquid crystal response delay may occur, generate a test image of high-frequency flicker, and observe the display of the area under different flicker frequencies, so as to judge whether there is a liquid crystal response delay failure;

[0064] Reinforcement learning optimization path: iteratively optimizing the test sequence through the Q-learning algorithm, shortening the detection time while ensuring the coverage rate (reducing redundant test steps by more than 30%), for example, in the initial test stage, the system randomly selects some test images for detection, calculates the reward value of each test image according to the detection results, and then continuously adjusts the test sequence through the Q-learning algorithm, preferentially selects test images with high reward values, thereby improving the detection efficiency;

[0065] Step 5: Fault diagnosis: Collect the inspection results of the display screen using an optimized test image sequence, comprehensively evaluate whether the display screen has any faults, as well as the type and severity of the faults, and produce a fault diagnosis report for the display screen. For example, by comparing the response of the display screen under different test images, determine whether there is a liquid crystal response delay or pixel sticking fault.

[0066] The multi-source signal synchronous acquisition method in step 1 is:

[0067] S1: Connect the optical sensor to the data acquisition host via a data cable (such as USB 3.0). The installation position of the optical sensor covers the visible area of ​​the display screen and avoids obstruction and reflection. Connect the electrical signal probe to the node of the display screen's drive circuit to ensure good electrical contact. The probe is connected to the data acquisition host via a shielded cable to reduce external electromagnetic interference. Adjust the focus and position of the thermal imaging module so that it focuses on the display screen surface. Transmit the thermal imaging data to the data acquisition host via the Ethernet interface. The environmental sensor is installed in the area where the display is located and communicates with the data acquisition host through the interface.

[0068] S2: Create a main thread to coordinate the data collection tasks of each sensor and create an independent sub-thread for each sensor;

[0069] S3: For the optical sensor thread, when the main thread receives the start acquisition instruction, the sub-thread begins to acquire image data from the camera at the set frame rate. In the electrical signal probe sub-thread, the voltage and current data transmitted by the probe are read in a loop and stored in a preset buffer. The thermal imaging module sub-thread acquires thermal imaging data at a set time interval (such as the number of frames per second, the specific time interval value). The environmental sensor sub-thread continuously collects ambient light and humidity data and updates the corresponding variables.

[0070] S4: Define the storage format for each sensor data. For example, image data is stored in JPEG format and the files are named according to the acquisition time. Voltage and current data are stored as text files or binary files. Each line records the relevant data of a time point, including the timestamp, voltage value, and current value.

[0071] The feature extraction method is:

[0072] S1: Optical feature extraction: Extract pixel statistical features and calculate statistical parameters such as the mean, variance, and standard deviation of the image. For example, for a grayscale image, the mean reflects the average brightness of the image, and the calculation formula is: Where M and N are the number of rows and columns of the image respectively, and P(i,j) represents the grayscale value of pixel (i,j);

[0073] The histogram features of the image are extracted, including the entropy, energy and contrast of the histogram. The entropy is used to measure the information richness of the image, and the calculation formula is: where p(k) is the probability of the pixel with the gray value k, and L is the total number of gray levels.

[0074] The texture features of the image are extracted. The gray level co-occurrence matrix (GLCM) is used to extract the texture features. The GLCM in different directions (such as 0°, 45°, 90° and 135°) is calculated, and the contrast, correlation, energy and uniformity features are extracted therefrom. For example, the contrast feature reflects the sharpness of the image texture, and the calculation formula is: i ∑ j (i-j) 2 P(i,j), where P(i,j) is the value of element (i,j) in the GLCM.

[0075] The texture features are extracted using the local binary pattern (LBP) operator. The LBP operator generates a binary code sequence to describe the local texture structure of the image by comparing the gray value of the center pixel with that of the surrounding pixels.

[0076] S2: Extraction of electrical signal features: Time domain features are extracted, and the effective value, average value and peak value of the voltage and current signals are calculated. The effective value is calculated by the formula: where x i is the voltage or current value of the sampling point, and N is the number of sampling points.

[0077] The pulse width, rise time and fall time parameters of the signal are extracted. These time parameters are determined by detecting the change of the signal edge. For example, the rise time is defined as the time required for the signal to rise from the low level to a certain threshold level.

[0078] Frequency domain features are extracted. The fast Fourier transform (FFT) is performed on the voltage and current signals to obtain the frequency spectrum distribution of the signals. The frequency component, amplitude and phase information are extracted from the frequency spectrum. For example, the frequency value and corresponding amplitude of the main frequency component, and the energy proportion of each frequency component are calculated.

[0079] S3: Extraction of thermal imaging features: Temperature statistical features are extracted. The average temperature, maximum temperature and minimum temperature of the thermal imaging image are calculated. The average temperature is obtained by adding the temperature values of all pixel points and dividing by the total number of pixels. The features of the temperature histogram are analyzed, such as the skewness and kurtosis of the temperature distribution. The calculation formula of the skewness is: where T i is the temperature value of the i-th pixel point, μ is the average temperature, and σ is the standard deviation of the temperature.

[0080] Extracting hotspot area features, detecting the hotspot area in the thermal imaging image by setting a temperature threshold, for example, marking the area with a temperature higher than a certain threshold as a hotspot area, and calculating the area, perimeter, position geometric features of the hotspot area;

[0081] S4: Obtain the light intensity value measured by the ambient light sensor and the humidity value measured by the humidity sensor as environmental features.

[0082] The construction of the multi-dimensional feature fault library in step three is as follows: integrate the data collected from various sensors and extracted features to form a data set containing multiple feature vectors, for example, for each set of collected data, combine its corresponding image feature vector, electrical signal feature vector, thermal imaging feature vector, and environmental feature vector; invite professional display screen fault diagnosis experts to label the collected data, including whether there is a fault and the type of fault (such as liquid crystal screen display abnormality, backlight failure, drive circuit failure); and select a database management system (such as MySQL, Oracle) to store the multi-dimensional fault feature library, design the database table structure, including the data table, the fault type table, and the sensor information table, the data table is used to store the collected data and the corresponding feature vector, the fault type table records the information of different fault types, and the sensor information table stores the related parameters and state information of the sensor.

[0083] Integrate the data collected from various sensors and extracted features to form a data set containing multiple feature vectors such as image, electrical signal, thermal imaging, and environment, this way can obtain the state information of the LCD display screen from different angles, for example, the image feature vector can reflect the pixel, texture, and other visual abnormalities of the screen; the electrical signal feature vector can reflect the driving voltage, current, and other circuit-related problems; the thermal imaging feature vector can show the temperature distribution abnormalities of different areas of the screen, and the environmental feature vector records the influence of environmental factors on the screen, combining these feature vectors is equivalent to providing evidence for fault diagnosis from multiple dimensions, greatly enriching the information source for fault judgment, making the judgment of display screen fault more comprehensive and accurate;

[0084] Invite professional display screen fault diagnosis experts to label the collected data, experts can accurately judge whether there is a fault and the type of fault, such as liquid crystal screen display abnormality, backlight failure, drive circuit failure, etc., this professional labeling gives the data in the multi-dimensional feature fault library accurate labels, making the subsequent analysis and fault judgment based on these data have a reliable reference standard, improving the accuracy and reliability of the entire fault diagnosis method;

[0085] The database management system is used to store the multi-dimensional fault feature library, which has good data storage and management capability, can efficiently store a large amount of feature vector data, labeling information and sensor related information, and can quickly retrieve the required data according to the user's needs. In the fault diagnosis process, when a specific type of fault or a group of sensor data needs to be queried, it can be quickly extracted from the database, providing strong support for timely and accurate fault judgment.

[0086] In this embodiment, a single sensor or data source can only provide limited information about the state of the LCD display screen, while multi-modal data can capture information in different aspects, for example, an optical sensor can detect image quality, an electrical signal probe can monitor voltage, current and other electrical parameters, a thermal imaging module can reflect temperature distribution, and an environmental sensor can provide environmental conditions affecting the display screen. By fusing these different modalities of data, a more comprehensive understanding of the working state of the LCD display screen can be obtained, reducing the possibility of missed detection and misjudgment. Different modalities of data can describe and analyze the LCD display screen from multiple angles, and through reasonable fusion algorithms, they can verify and supplement each other, thereby improving the accuracy of fault judgment.

[0087] Multi-modal data dynamic fusion can process data collected by different sensors in real time, update the evaluation of the state of the LCD display screen in a timely manner, and achieve rapid and accurate fault judgment. For some occasions with high real-time requirements, such as LCD display screen detection on industrial automation production lines, faults can be discovered and handled in a timely manner, avoiding the production of a large number of defective products.

[0088] Different modalities of data describe and analyze the display screen from multiple angles, and verify and supplement each other. For example, a suspected defect area found by an optical sensor can be detected by an electrical signal probe to determine whether the circuit in that area is normal, and a thermal imaging module can be used to check whether there is overheating in that area, thereby more accurately determining whether there is a real defect in that area. The multi-modal data dynamic fusion detection technology has shown significant advantages and benefits in the LCD display screen picture quality detection system patent, not only improving the comprehensiveness, accuracy and efficiency of detection.

[0089] Embodiment 2

[0090] This embodiment is an optimization based on embodiment 1, specifically, the multi-source signal synchronous acquisition and preprocessing in step one includes a time sequence correlation module, which includes a hardware clock synchronization unit, a software timestamp calibration unit and an association rule mining unit.

[0091] Hardware clock synchronization unit: In the multi-source data acquisition module, a hardware clock synchronization device is used, such as a network time protocol (NTP) server or a precise clock source (such as a GPS clock), which synchronizes the device clock of each sensor with the standard time server through the NTP protocol, ensuring the accuracy and consistency of the time stamp; and a timer interrupt program is set in the multi-source data acquisition module, which sends a synchronization signal to each sensor simultaneously whenever the timer triggers, so that each sensor records data and stamps the time at the same time;

[0092] Software time stamp calibration unit: In the multi-source data acquisition module, for each collected data sample, record its collection time stamp, and calibrate the data collection time stamps of different sensors to make them have the same frequency, such as assuming that the frame rate of the optical sensor is 30fps, the sampling rate of the electrical signal probe is 1kHz, and the frame rate of the thermal imaging module is 10fps, when a group of data is collected, find the smallest time unit (such as milliseconds) between them, and convert the data time stamps of each sensor to multiples of this unit, if the time stamp of the optical sensor collecting a frame of image is t1, the electrical signal probe collects multiple data points at t1, and the data point closest to t1 is taken as the time stamp data of the optical image corresponding to the electrical signal; For the time delay error, interpolation or curve fitting method is used for correction, for example, by analyzing the data transmission delay law between sensors, a function relationship model of delay time and sensor data is established, and the time stamp is corrected;

[0093] Association rule mining unit: Apriori algorithm is used to find the association rules between sensor data, first scan the data set to find all the frequently occurring single sensor data features (i.e. features that meet the minimum support threshold), then gradually find the frequently occurring multi-item sensor data combination features through iteration, and finally generate the association rules;

[0094] The algorithm implementation steps of Apriori algorithm are:

[0095] S1: Data acquisition and integration: Collect data from optical sensors, electrical signal probes, thermal imaging modules and environmental sensors, assuming that n groups of data are collected, each group of data contains optical sensor data O i (image feature vector), electrical signal probe data E i (voltage, current value), thermal imaging data T i (temperature distribution information) and environmental sensor data H i(environmental light, humidity values), pre-process the collected data, including data cleaning (remove outliers, noise), standardization (convert data of different modalities to a unified dimension or scale), and discretization (convert continuous data to discrete intervals or categories), for example, for voltage data of the electrical signal probe, divide it into low, medium, and high intervals according to its value range;

[0096] S2: Transaction database construction: each set of pre-processed data is regarded as a transaction, for example, the jth transaction is represented as T = {O j ,E j ,T j ,H j}, where O j is the presence or absence of certain features in the image (such as the presence or absence of bright spot defect features), E j is a set of electrical signal intervals, T j is a set of temperature intervals, and H j is a set of environmental parameter intervals, so that all transactions constitute a transaction database D;

[0097] S3: Frequent item set mining: including frequent item set mining of single sensor data features and frequent item set mining of multi-item sensor data combination features;

[0098] Frequent item set mining of single sensor data features (L1): set the minimum support threshold min-sup, for each sensor modality data feature, count the number of times it appears in the data set, for example, for the feature o of the optical sensor, calculate the frequency count(o) of its appearance in all transactions, if then add feature o to the frequent 1-item set L1, and perform the same operation on the features of the electrical signal probe, thermal imaging, and environmental sensors to obtain their respective initial frequent 1-item sets;

[0099] Frequent item set mining of multi-item sensor data combination features (L k ,k≥2): assuming that the frequent (k-1)-item set L k -1 has been obtained, generate a set of candidate k-item sets C k through self-connection operation, for example, for two frequent (k-1)-item sets l1 and l2, if their first (k-2) features are the same, then they are combined to generate a candidate k-item set, for each candidate k-item set c, calculate its support: where count(c) is the number of transactions containing c, if Support(c) ≥ min-sup, then add c to the frequent k-item set L k ; repeat the above steps until no new frequent item set can be generated, i.e. all frequent item sets L = L1∪L2∪…∪Lm ;

[0100] S4: Association rule generation: For each frequent itemset l e L (assume l contains k features), generate all possible non-empty subset combinations as the Left Hand Side (LHS) and Right Hand Side (RHS) of rules, for example, for the frequent 2-itemset {o, e} (where o is an optical sensor feature and e is an electrical signal probe feature), generate rules o→e and e→o, calculate the confidence of each rule: If the confidence meets a given minimum confidence threshold min_conf, keep the rule;

[0101] S5: Rule optimization: Remove redundant rules, if there are two rules R1: A→B and R2: A→C, and The confidence of both rules is similar, remove rule R2 because R1 can already cover the part of the cases that R2 covers, adjust the rules according to expert experience or actual application scenarios, for example, in certain specific display screen working environments, some rules may not be applicable, and these rules need to be modified or deleted;

[0102] Also consider the timeliness and order of the data:

[0103] Time window division: According to actual needs and data characteristics, select an appropriate time window size, for example, divide the time window by every 10 seconds or every minute, in each time window, only the sensor data falling within the window is analyzed for association, ensuring that the association rules are based on the data relationship in the same time period, and more consistent with the actual time sequence logic;

[0104] Sequential data processing: When mining frequent itemsets, not only consider whether each feature appears at the same time, but also consider the order in which they appear, this can be achieved by adding an order label to each feature in a transaction, for example, for a transaction T = {O1, E2, T3, H4} (where the numbers represent the order in which the features appear), when generating frequent itemsets and association rules, the order factor should be considered first, for example, the rule O1→E2 (which means that when the optical sensor feature 1 appears first, the electrical signal probe feature 2 also appears) and the rule E2→O1 (which means that when the electrical signal probe feature 2 appears first, the optical sensor feature 1 also appears) may have different confidence and support, and need to be calculated and evaluated separately);

[0105] Rule evaluation and screening: evaluate the mined association rules, calculate their support and confidence indicators, remove those rules with too low support or confidence, and retain strong association rules as the basis for fault judgment. According to the actual application scenario and expert experience, the association rules are screened and optimized, for example, some association rules may only be applicable in specific display screen models or working environments, and need to be adjusted and improved according to the specific situation.

[0106] In this embodiment, by using a Network Time Protocol (NTP) server or a precise clock source (such as a GPS clock), and synchronizing the device clocks of each sensor with the standard time server using the NTP protocol, the timestamps from different sensors can be ensured to be accurate and consistent, which is crucial for subsequent fault analysis, because accurate time information can accurately locate the time of fault occurrence, avoiding misjudgment of the fault cause due to inaccurate time. A timer interrupt program is set, and a synchronization signal is sent to each sensor whenever the timer triggers, so that each sensor records data and timestamps at the same time, which further enhances the synchronization of data collection, especially in the case of real-time correlation analysis of data from multiple sensors;

[0107] For sensors with different sampling rates and frame rates, such as optical sensors (30fps), electrical signal probes (1kHz), and thermal imaging modules (10fps), the data timestamps of these sensors are uniformly converted into multiples of the smallest time unit (such as milliseconds), so that the data of different sensors can be analyzed on the same time scale. This helps to integrate multi-source data and mine potential relationships between data. Interpolation or curve fitting methods are used to correct the existing time delay errors, taking into account the data transmission delay rules between sensors, and a function relationship model between delay time and sensor data is established to correct the timestamps, which can improve the reliability and accuracy of the data and avoid data misplacement caused by time delay;

[0108] The Apriori algorithm is used to discover association rules between sensor data, which can mine valuable information hidden in the data. By scanning the data set to find frequently occurring single sensor data features, and then gradually finding multi-item sensor data combination features, association rules are finally generated. This helps to understand the mutual relationship between different sensor data, providing new ideas for fault judgment. In the LCD display screen display fault judgment, through association rule mining, it may be found that when the optical sensor detects abnormal brightness in a specific area, the electrical signal probe detects voltage fluctuations in that area beyond a certain range, and the thermal imaging module displays temperature rise in that area, there is a high probability of a bad point fault in the display screen. This association rule can help technicians quickly locate the fault cause and improve the efficiency of fault elimination.

[0109] Embodiment 3

[0110] This embodiment is based on Example 1 or Example 2 and the following optimizations are made, specifically, the dynamic test pattern generation in step four includes a preliminary detection module, a dynamic test image generation module and a reinforcement learning optimization module;

[0111] Preliminary detection module: use optical sensors to collect the initial image of the display screen, pre-process the image (such as noise reduction, contrast enhancement), and then use feature extraction algorithms (such as edge detection, pixel value statistical analysis) to preliminarily judge the areas that may have faults, for example, use edge detection algorithm to find out whether there are obvious bright or dark spot abnormal areas in the display screen;

[0112] Dynamic test image generation module: according to the preliminary detection results, generate targeted test images through deep neural networks, this module contains multiple sub-networks for generating different types of test images, including high-frequency flicker image generation sub-network, gradual gray scale image generation sub-network, polar coordinate grid image generation sub-network, each sub-network receives the output results of the preliminary detection module, including possible fault area location and type information, and then generates corresponding test images;

[0113] Reinforcement learning optimization module: use Q-learning algorithm to optimize the generated test image sequence, this module records the execution effect of each test image (evaluated by reward value), and adjusts the order and selection of the test sequence according to these records to shorten the detection time while ensuring coverage;

[0114] Data interaction process:

[0115] S1: The preliminary detection module transmits the processed preliminary detection results (possible fault area information) to the dynamic test image generation module;

[0116] S2: The dynamic test image generation module generates an initial test image sequence according to the received information, and provides these test images to the display screen for testing, while passing the information during the test process (such as test image type, display effect, etc.) to the reinforcement learning optimization module;

[0117] S3: The reinforcement learning optimization module calculates the reward value of each test image according to the received information, and updates the test sequence through Q-learning algorithm, the optimized test sequence is fed back to the dynamic test image generation module for guiding the generation of the next round of test images;

[0118] The neural network structure of the high-frequency flicker image generation sub-network includes:

[0119] Input layer: receives possible fault area location and type information, divides the display screen into m x n areas, and uses an m x n vector to represent whether each area is suspected of having a fault (1 for yes and 0 for no);

[0120] Hidden layer: adopts a multi-layer perceptron (MLP) structure, contains h1 neurons, and the activation function is ReLU. The output of the input layer is transmitted to the hidden layer through full connection, and the calculation formula is: z (1) = W (1) x + b (1) , a (1) = ReLU(z (1) ); where W (1) is the weight matrix from the input layer to the hidden layer, b (1) is the bias vector, and x is the input vector;

[0121] Output layer: outputs the pixel value matrix of the high-frequency flicker image. For each pixel point (i, j), the output layer calculates its gray value y ij : where, is the weight matrix from the hidden layer to the output layer, is the bias vector, and σ is the sigmoid activation function, which is used to map the pixel value to between 0 and 1;

[0122] The training process of the high-frequency flicker image generation subnetwork is as follows:

[0123] Prepare a number of display screen image data with annotations, including fault area location and type. Use 70% of the data as the training set and 30% of the data as the validation set. For each training sample, input the annotated fault area information into the neural network, calculate the difference (such as mean square error) between the output high-frequency flicker image and the actual annotated flicker image, adjust the weights and biases of the neural network through the backpropagation algorithm to minimize the difference, and verify the performance of the trained neural network on the validation set. Adjust the hyperparameters (such as learning rate and number of hidden layer neurons) based on the validation results. For some faults that are difficult to detect under normal display but are easily exposed under specific frequency flicker, such as poor contact of some lines, small pixel point abnormalities, etc., the high-frequency flicker image can make them more easily detected, because these faults may not be obvious under static or regular dynamic display, but high-frequency flicker can stimulate fault characteristics, allowing detection personnel or systems to more accurately capture fault signs;

[0124] The neural network structure of the gradual gray scale image generation subnetwork includes:

[0125] Input layer: receives fault area information, divides the display screen into m x n areas, and uses an m x n vector to represent whether each area is suspected of having a fault (1 for yes and 0 for no);

[0126] Hidden layer: uses a convolutional neural network (CNN) structure, which contains k convolution kernels, each of which is c×c in size. The convolution operation extracts features from the input data. The calculation formula is: Among them, W is the convolution kernel weight matrix, x is the input data (fault area information), b is the bias vector, Represents the convolution operation;

[0127] Output layer: Outputs the grayscale matrix of the gradient grayscale image. For each pixel (i, j), the output layer calculates its grayscale value: y ij =W' ij a+b' ij , where W' ij is the weight matrix from the hidden layer to the output layer, b' ij is the bias vector;

[0128] The training process of the gradient grayscale image generation sub-network is:

[0129] Several display screen images with different fault types are collected as training data. The fault area information is extracted from each image and the corresponding gradient grayscale image is labeled. A supervised learning method is used to train the neural network so that the output gradient grayscale image is as close as possible to the labeled image. The cross-entropy loss function is used during the training process to measure the difference and adjust the model parameters.

[0130] The grayscale display capability of LCD screens is one of their most important performance indicators. Gradient grayscale images can help detect the screen's performance when transitioning between different brightness levels. Faults, such as abnormal liquid crystal alignment or driver circuit failure, can result in uneven grayscale transitions, streaks, or color blocks. This targeted test image can accurately detect such grayscale display-related faults.

[0131] The neural network structure of the polar coordinate grid image generation subnetwork includes:

[0132] Input layer: Receives the location and size information of the fault area. For example, a four-dimensional vector is used to represent the center coordinates (x, y) and radius r of each fault area.

[0133] Hidden layer: It uses a recurrent neural network (RNN) structure, which contains l neurons. RNN processes sequence data and generates a sequence of polar coordinate grid images based on the input fault area information. The output of the input layer is passed to the hidden layer according to the time step. The state update formula of the hidden layer is: h t =σ(Wh t-1 +U xt +b), where h tis the state of the hidden layer at time t, W is the weight matrix of the hidden layer, U is the weight matrix of the input to the hidden layer, xtis the input vector at time t, b is the bias vector, and σ is the activation function;

[0134] Output layer: convert the state of the hidden layer into a polar coordinate grid image pixel matrix, for each pixel point (i, j), calculate whether its position in polar coordinates is within the fault area according to the hidden layer state, if it is, set it to a specific value (such as 1), otherwise set it to another value (such as 0);

[0135] The training process of the polar coordinate grid image generation subnetwork is as follows:

[0136] Construct a training data set containing fault area information and corresponding polar coordinate grid image labels, serialize the input data and input it into the RNN for training, and during the training process, evaluate the model performance by calculating the accuracy between the predicted polar coordinate grid image and the labeled image, and adjust the model parameters according to the loss value;

[0137] Such images are helpful for detecting geometric deformation of the display screen, pixel arrangement regularity, etc., for example, if the display screen is squeezed by external force causing local deformation, or the internal pixel structure is dislocated, etc., in the polar coordinate grid image, it will show abnormal grid distortion, deformation, etc., which is convenient for detection personnel to intuitively find such faults related to the physical structure and pixel layout of the display screen;

[0138] The Q-learning algorithm optimizes the generated test image sequence in the following way:

[0139] S1: State definition: the state space S consists of the following elements: the state information of the current display screen area to be detected, including the results of the detected areas (whether there is a fault), the characteristics of the remaining undetected areas (such as area, position);

[0140] The history record of the selected test image sequence, including the type of the test image, the execution order and the detection result (reward value) of each test image in this state, for example, divide the display screen into 9 areas, and the first 3 areas have been detected, among which the first area has a fault, and the second and third areas are normal; the selected test image sequence is [high-frequency flicker image, gradual gray scale image], the reward value of the high-frequency flicker image in the first area is 5, and the reward value in other areas is 0; the reward value of the gradual gray scale image in all areas is -1. Then the current state can be represented as: S = ([1, 0, 0, *, *, *, *, *, *]), [(high-frequency flicker image, [5, 0, 0])], [(gradual gray scale image, [-1, -1, -1])], where * represents that the area has not been detected;

[0141] S2: Action definition: the action space A includes selecting the next test image to be executed, assuming there are n different test images (such as high-frequency flicker, gradual gray scale, polar coordinate grid), then the action set is: A = {a1, a2, …, an}, after selecting an action each time, the corresponding test image is executed and its effect is observed (the reward value is obtained); n};

[0142] S3: Reward function design: the reward function R(s, a, s') is determined according to the change of the state after executing the action, if the test image discovers a new fault area, a higher positive reward is given, for example, the reward value is 10, if the test image excludes the suspicion of a large area of fault-free area, a certain positive reward is given, such as the reward value is 3, if the test image does not provide effective new information (neither discovers new faults nor excludes more fault-free areas), a smaller negative reward is given, such as -1, if the test image misjudges the fault area (i.e. judges the area with faults as fault-free, or vice versa), a larger negative reward is given, such as -5, the specific form of the reward function is adjusted and optimized according to actual needs, for example: R(s, a, s') = w1 x new fault area discovered - w2 x misjudged fault area + w3 x excluded fault-free area; wherein w1, w2, w3 are weight coefficients, which are determined according to experience or experiment;

[0143] S4: Q table update: initialize the Q table, set the Q value of all state-action pairs to 0 or a small random value, update the Q table according to the actual reward after executing the action and the estimated future reward, for each state-action pair (s, a), the target Q value Q'(s, a) can be calculated by the following formula: Q'(s, a) = R(s, a, s') + γmax a'∈A Q(s', a'), wherein γ is a discount factor (0≤γ<1), used to balance the importance of immediate reward and future reward; then update the actual Q value according to the target Q value: Q(s, a) = Q(s, a) + α[Q'(s, a) - Q(s, a)], wherein α is the learning rate;

[0144] S5: policy improvement: select the optimal policy according to the updated Q table, in each state, select the action with the highest Q value as the optimal action, for example, in state S, if max a∈A Q(s, a) = Q(s, a3), then select action a3 as the optimal policy, with the training and the change of the environment (such as the change of the display screen fault type, the change of the test image effect), constantly adjust the Q table and the policy to adapt to the new detection task), in the training process, the above steps are executed iteratively until the Q table converges or the preset training number of rounds is reached.

[0145] In this embodiment, the initial image of the display screen is collected by the optical sensor, the information of the actual display state of the screen can be directly obtained, the image is preprocessed and preliminarily analyzed through the feature extraction algorithm, and the area where the fault may exist can be rapidly screened out, which greatly reduces the range of subsequent key detection, avoids the complex detection of the entire display screen, improves the detection efficiency, and improves the detection efficiency.

[0146] Each sub-network receives the output result of the preliminary detection module and generates a corresponding test image, and multiple different types of sub-networks cooperate with each other to detect the area where the fault may exist from different angles, whether it is an electrical performance fault (which can be detected by high-frequency flicker), a gray scale display capability fault (which is detected by means of a gradual gray scale), or a physical structure and pixel arrangement fault (which is detected by means of a polar coordinate grid), all of which can be covered, thereby greatly improving the comprehensiveness and accuracy of the display screen fault detection.

[0147] The Q-learning algorithm is adopted to optimize the generated test image sequence, the execution effect of each test image is recorded, and the order and selection of the test sequence are adjusted according to the records, so that the test images that have little contribution to fault judgment or have high repetition can be adjusted or removed, the entire detection process can be focused on the key and effective test images, the detection time is shortened, and the method is especially suitable for the case of rapidly detecting a large number of display screens on a large-scale production line.

[0148] Please refer to Figure 2 The application further discloses an LCD display screen display fault judgment system, which comprises a multi-source data acquisition module, a central processor, a memory and a user interface.

[0149] The multi-source data acquisition module includes an optical sensor, an electrical signal probe, a thermal imaging module, an environmental sensor, and a data acquisition host; the optical sensor is a camera, which is used to collect image data of the LCD display screen, to obtain the display content, brightness, and color information of the screen, so as to analyze whether there are pixel point faults, line faults, and abnormal color block display problems; the electrical signal probe is a voltage and current probe, which monitors the driving voltage and current of the LCD display screen, can obtain the waveform, amplitude, and frequency of the electrical signal in real time, so as to determine whether the power supply circuit is normal and whether the signal transmission is stable, and is helpful to find display abnormalities caused by circuit faults; the thermal imaging module is a thermal imager, which detects the temperature distribution of each area of the LCD display screen through thermal imaging technology, can timely find hidden troubles caused by overheating or poor heat dissipation, such as the problem of uneven backlight or damage caused by the excessively high temperature of the backlight plate; the environmental sensor is a light sensor and a humidity sensor, which collects environmental light intensity and humidity data, and the excessively strong or weak environmental light may affect the visibility and contrast effect of the display screen, and the change of humidity may have a bad influence on the corrosion of the electronic components inside the display screen, and these environmental factors may indirectly cause display faults or affect the display effect;

[0150] The central processing unit is the core control unit of the system, is responsible for coordinating the work of various sensors, receiving the data transmitted by the sensors, and performing preliminary processing and analysis, running the fault judgment algorithm, comparing and judging the data according to the preset rules and models, to determine whether the LCD display screen has faults and the type and severity of the faults;

[0151] The memory is used to store system programs, fault feature libraries, and sensor calibration parameter data, the system program is a software instruction set for controlling the operation of the entire system; the fault feature library contains the characteristic information of various common faults and the corresponding processing measures; the sensor calibration parameter is used to ensure the accuracy and reliability of the sensor measurement;

[0152] The user interface is used to set system parameters, start and stop data acquisition and fault judgment process, view fault judgment results and historical records, realize automatic control and operation management of the system, and automatically control the working state and data acquisition process of each hardware component according to the preset conditions and rules.

[0153] To sum up: the present application synchronously collects various data including display data, driving voltage / current monitoring data, temperature distribution and environmental parameters, etc. by optical sensors, electrical signal probes, thermal imaging modules and environmental sensors, which can obtain all-around information of the LCD display during operation. The collection of multi-source data provides a solid foundation for subsequent accurate fault judgment, avoiding the limitations and misjudgments that may be caused by a single data source. The pretreatment operations such as denoising, adjusting contrast and brightness, etc. can effectively improve the quality of data, reduce noise interference, make the data more accurate and clear, and be beneficial to subsequent feature extraction and fault judgment. At the same time, the storage of pretreated data also provides convenient data support for subsequent analysis.

[0154] A multi-dimensional feature fault library containing a plurality of fault cases is constructed, and the fault phenomena are associated with the underlying causes, which provides valuable reference for fault judgment. In actual fault diagnosis, the type and cause of the fault can be quickly and accurately judged by comparing and analyzing the cases in the fault library, which improves the efficiency and accuracy of fault diagnosis.

[0155] According to the preliminary detection result, the targeted test image is dynamically generated, which can stimulate potential faults and make potential problems more easily exposed. This method of dynamically generating test images can be adjusted and optimized according to the actual situation, which improves the effectiveness and coverage of the test. The test sequence is iteratively optimized by the Q-learning algorithm, which shortens the detection time while ensuring the coverage. The reinforcement learning algorithm can automatically adjust the strategy according to the environmental feedback, making the test process more intelligent and efficient. By continuously optimizing the test sequence, the display screen can be comprehensively and accurately detected in the shortest time. The detection results of the display screen by the optimized test image sequence are collected to comprehensively evaluate whether the display screen has faults and the type and severity of the faults, and a fault diagnosis report is obtained.

[0156] Finally, it should be noted that: the above only describes the preferred embodiments of the present application and is not used to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent replacements to some technical features, as long as they are within the spirit and principles of the present application. Any modification, equivalent replacement, improvement, etc. shall be included in the protection scope of the present application.

Claims

1. A method for judging LCD display faults, characterized in that: The following steps are involved: Step 1: Synchronous multi-source signal acquisition and preprocessing: Optical sensors, electrical signal probes, thermal imaging modules, and environmental sensors are used to synchronously acquire LCD display data, drive voltage / current monitoring data, temperature distribution, and environmental parameters. The acquired data is preprocessed, including noise removal, contrast and brightness adjustment, and stored. Step 2: Feature extraction: Extract key features from the preprocessed image, including pixel and texture features, analyze changes in electrical signals, temperature distribution, and environmental parameters, and extract feature information related to the fault; Step 3: Establish a fault diagnosis model: Build a multi-dimensional feature fault library containing several fault cases, and associate the fault phenomenon with the underlying cause; Step 4: Dynamic test pattern generation: Based on the preliminary test results, dynamically generate targeted test images to stimulate potential faults; Specifically, it includes a preliminary detection module, a dynamic test image generation module and a reinforcement learning optimization module; the preliminary detection module is used to preliminarily determine the area where the fault may exist; the dynamic test image generation module includes a high-frequency flicker image generation subnetwork, a gradient grayscale image generation subnetwork, and a polar coordinate grid image generation subnetwork, each of which is used to generate a corresponding test image based on the preliminary detection results of the preliminary judgment; The reinforcement learning module is used to strengthen the learning optimization path: it uses the Q-learning algorithm to iteratively optimize the test sequence, shortening the detection time while ensuring coverage; Step 5: Fault diagnosis: Collect the detection results of the display screen using the optimized test image sequence, comprehensively evaluate whether the display screen has a fault, the type and severity of the fault, and obtain a fault diagnosis report for the display screen.

2. The method for determining an LCD display screen display failure according to claim 1, wherein: The multi-source signal synchronous acquisition method described in step 1 is: S1: Connect the optical sensor to the data acquisition host via a data cable. The installation position of the optical sensor covers the visible area of ​​the display screen. Connect the electrical signal probe to the node of the display screen's drive circuit. The probe is connected to the data acquisition host via a shielded cable. Adjust the focus and position of the thermal imaging module so that it focuses on the display screen surface. Transmit the thermal imaging data to the data acquisition host via the Ethernet interface. The environmental sensor is installed in the area where the display is located and communicates with the data acquisition host through the interface. S2: Create a main thread to coordinate the data collection tasks of each sensor and create an independent sub-thread for each sensor; S3: For the optical sensor thread, when the main thread receives the start acquisition instruction, the sub-thread begins to acquire image data from the camera at the set frame rate. In the electrical signal probe sub-thread, the voltage and current data transmitted by the probe are read in a loop and stored in a preset buffer. The thermal imaging module sub-thread acquires thermal imaging data at a set time interval. The environmental sensor sub-thread continuously collects ambient light and humidity data and updates the corresponding variables. S4: Define the storage format for each sensor data.

3. The method for determining an LCD display screen fault according to claim 1, wherein: The feature extraction method described in step 2 is: S1: Optical feature extraction: extract pixel statistical features and calculate the mean, variance, and standard deviation statistical parameters of the image; S2: Electrical signal feature extraction: Extract time domain features and calculate the effective value, average value, and peak value of voltage and current signals; S3: Thermal imaging feature extraction: Extract temperature statistical features and calculate the average temperature, maximum temperature, and minimum temperature of the thermal imaging image. The average temperature is obtained by adding the temperature values ​​of all pixels and dividing it by the total number of pixels. Analyze the characteristics of the temperature histogram. S4: Acquire the light intensity value measured by the ambient light sensor and the humidity value measured by the humidity sensor as environmental features.

4. The method for determining an LCD display screen fault according to claim 1, wherein: The multi-dimensional feature fault library described in step 3 is constructed by integrating the data collected and extracted from each sensor to form a data set containing multiple feature vectors, and annotating the collected data to include whether there is a fault and the type of fault; A database management system is selected to store the multidimensional fault feature library, and the database table structure is designed, including a data table, a fault type table, and a sensor information table. The data table is used to store the collected data and the corresponding feature vectors. The fault type table records information of different fault types. The sensor information table stores the relevant parameters and status information of the sensor.

5. The method for determining an LCD display screen display failure according to claim 1, wherein: The multi-source signal synchronous acquisition and preprocessing described in step 1 includes a timing association module, which includes a hardware clock synchronization unit, a software timestamp calibration unit and an association rule mining unit; Hardware clock synchronization unit: A hardware clock synchronization device is used in the multi-source data acquisition module to synchronize the device clock of each sensor with the standard time server through the NTP protocol. A timer interrupt program is set in the multi-source data acquisition module. Whenever the timer is triggered, a synchronization signal is sent to each sensor at the same time, so that each sensor records data at the same time and adds a time stamp. Software timestamp calibration unit: In the multi-source data acquisition module, for each collected data sample, the acquisition timestamp is recorded and the data acquisition timestamps of different sensors are calibrated to have the same frequency; Association rule mining unit: uses the Apriori algorithm to discover association rules between sensor data.

6. The method for determining an LCD display screen fault according to claim 5, wherein: The algorithm implementation steps of the Apriori algorithm are: S1: Data acquisition and integration: Collect data from optical sensors, electrical signal probes, thermal imaging modules, and environmental sensors, and pre-process the collected data, including data cleaning, standardization, and discretization; S2: Transaction database construction: each set of pre-processed data is regarded as a transaction; S3: Frequent item set mining: including frequent item set mining of single sensor data features and frequent item set mining of multiple sensor data combination features; S4: Association rule generation: For each frequent item set, generate all possible non-empty subset combinations as the antecedents and consequents of the rule, generate rules, and calculate the confidence of each rule; if the confidence meets the given minimum confidence threshold, the rule is retained; S5: Rule optimization: remove redundant rules.

7. The method for determining an LCD display screen fault according to claim 1, wherein: The generation of the dynamic test pattern described in step 4 specifically includes: Preliminary detection module: uses an optical sensor to capture the initial image of the display screen, preprocesses the image, and then uses a feature extraction algorithm to preliminarily determine the area where the fault may exist; dynamic test image generation module: based on the preliminary detection results, a deep neural network is used to generate targeted test images. This module contains multiple sub-networks for generating different types of test images, including a high-frequency flicker image generation sub-network, a gradient grayscale image generation sub-network, and a polar coordinate grid image generation sub-network. Each sub-network receives the output results of the preliminary detection module, including the possible location and type information of the fault area, and then generates the corresponding test image; reinforcement learning optimization module: uses the Q-learning algorithm to optimize the generated test image sequence. This module records the execution effect of each test image and adjusts the order and selection of the test sequence based on these records to shorten the detection time while ensuring coverage.

8. The method for determining an LCD display screen fault according to claim 7, wherein: The training process of the high-frequency flicker image generation subnetwork is as follows: a number of display screen image data with annotations including the location and type of the fault area are prepared, 70% of the data is used as a training set, and 30% of the data is used as a validation set; for each training sample, the annotated fault area information is input into the neural network, the difference between the output high-frequency flicker image and the actual annotated flicker image is calculated, and the weights and biases of the neural network are adjusted through the back-propagation algorithm to minimize the difference; The training process of the gradient grayscale image generation subnetwork is as follows: a number of display screen images with different fault types are collected as training data, the fault area information is extracted from each image and the corresponding gradient grayscale image is labeled, and the neural network is trained using a supervised learning method to make the output gradient grayscale image as close as possible to the labeled image. During the training process, a cross entropy loss function is used to measure the difference and adjust the model parameters; The training process of the polar coordinate grid image generation subnetwork is as follows: constructing a training dataset containing fault area information and corresponding polar coordinate grid image annotations, serializing the input data and inputting it into the RNN for training. During the training process, the model performance is evaluated by calculating the accuracy between the predicted polar coordinate grid image and the annotated image, and the model parameters are adjusted according to the loss value.

9. The method for determining an LCD display screen fault according to claim 7, wherein: The Q-learning algorithm optimizes the generated test image sequence in the following way: S1: State definition: The state space S consists of the following elements: the state information of the display area to be inspected, including the results of the inspected area and the characteristics of the remaining uninspected area; the history of the selected test image sequence, including the type of test image, the execution order, and the inspection result of each test image in the state; S2: Action definition: The action space A includes selecting the next test image to be executed. After each action is selected, the corresponding test image is executed and its effect is observed; S3: Reward function design: The reward function R is determined based on the change in state after the action is executed. If the test image finds a new fault area, a positive reward is given. If the test image misidentifies the fault area, a negative reward is given. S4: Q-table update: Initialize the Q-table, set the Q-values ​​of all state-action pairs to 0 or a small random value, and update the Q-table based on the actual reward after executing the action and the estimated future reward; S5: Strategy improvement: Select the optimal strategy based on the updated Q-table. In each state, the action with the highest Q-value is selected as the optimal action.

10. An LCD display screen fault diagnosis system, characterized by: The system comprises a multi-source data acquisition module, a central processing unit, a memory and a user interface; the system realizes an LCD display screen display fault judgment method according to any one of claims 1 to 9 by combining the multi-source data acquisition module, the central processing unit, the memory and the user interface; The multi-source data acquisition module includes an optical sensor, an electrical signal probe, a thermal imaging module, an environmental sensor, and a data acquisition host. The optical sensor is a camera, which is used to collect image data of the LCD display and obtain the display content, brightness, and color information of the screen. The electrical signal probe is a voltage and current probe, which monitors the driving voltage and current of the LCD display. The thermal imaging module is a thermal imager, which detects the temperature distribution of each area of ​​the LCD display through thermal imaging technology. The environmental sensors are a light sensor and a humidity sensor, which collect ambient light intensity and humidity environment data. Central Processing Unit: Responsible for coordinating the work of various sensors, receiving data transmitted by sensors, and performing preliminary processing and analysis. It also runs fault diagnosis algorithms and compares and judges data according to preset rules and models to determine whether there is a fault on the LCD display and the type and severity of the fault. Memory: used to store system programs, fault signature database, and sensor calibration parameter data. The system program is the software instruction set that controls the operation of the entire system; the fault signature database contains characteristic information of various common faults and corresponding treatment measures; the sensor calibration parameters are used to ensure the accuracy and reliability of sensor measurements; User interface: used to set system parameters, start and stop data collection and fault diagnosis processes, view fault diagnosis results and historical records, realize automatic control and operation management of the system, and automatically control the working status of each hardware component and data collection process according to preset conditions and rules.

Images