Method and system for judging display fault of LCD (liquid crystal display) screen

Abstract

The invention discloses an LCD display screen display fault judgment method, and relates to the related technical field of LCD display screen detection. Comprising the steps of synchronous acquisition and preprocessing of multi-source signals, feature extraction, establishment of a fault judgment model, generation of a dynamic test mode, dynamic generation of a targeted test image according to a preliminary detection result to excite a potential fault, iterative optimization of a test sequence by a Q-learning algorithm, and fault diagnosis. The invention also discloses a system for judging the display fault of the LCD screen. The system comprises a multi-source data acquisition module, a central processing unit, a memory and a user interface. Various data including display data, driving voltage / current monitoring data, temperature distribution, environment parameters and the like are synchronously collected through the optical sensor, the electric signal probe, the thermal imaging module and the environment sensor, all-directional information of the LCD display screen in the operation process can be obtained, and limitation and misjudgment possibly caused by a single data source are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field related to the detection of LCD display screens, and particularly relates to a method and system for judging display faults of LCD display screens. Background Art

[0002] With the rapid development of modern technology, liquid crystal display (LCD) screens, as important interfaces for human-computer interaction, have been widely used in various electronic devices, such as smart phones, tablets, TVs, computer monitors, etc. However, due to the influence of long-term use, environmental factors, and manufacturing processes, various faults may occur in LCD display screens. Common faults include pixel damage, abnormal display, color distortion, response time delay, etc. These faults not only affect the user's visual experience but may also cause the device to malfunction. Even in some critical scenarios, such as the medical and aviation fields, display screen faults may also lead to serious safety problems.

[0003] Currently, traditional methods for judging LCD display screen faults mainly rely on manual inspection and simple test images or observing the screen display to determine whether there are problems. This method has many deficiencies: on the one hand, manual inspection is easily affected by personal subjective judgment, and different inspectors may obtain different results, leading to misjudgment or omission of problems; on the other hand, some existing technologies only use a single detection method, such as only judging whether there are faults on the screen through image comparison analysis. For some complex fault types or early potential faults, it is difficult to accurately detect and diagnose. For example, relying only on visual inspection or simple electrical signal testing may not be able to detect deep-level problems such as internal circuit aging and abnormal liquid crystal molecule arrangement.

[0004] Therefore, it is very necessary to propose a method and system for judging LCD display screen faults to solve the above problems. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention provides a method and system for judging LCD display screen faults, which solves the problems that different inspectors may obtain different results in the existing fault judgment methods, leading to misjudgment or omission of problems, and it is difficult to accurately detect and diagnose some complex fault types or early potential faults only through image comparison analysis.

[0006] To achieve the above object, the present invention is realized through the following technical solutions:

[0007] A method for judging LCD display screen faults includes the following steps:

[0008] Step 1: Multi-source signal synchronous acquisition and preprocessing: Use optical sensors, electrical signal probes, thermal imaging modules, and environmental sensors to synchronously acquire the display data, drive voltage / current monitoring data, temperature distribution, and environmental parameters of the LCD display screen. Preprocess the acquired data, including denoising, adjusting contrast and brightness, ensuring the quality and consistency of the images, and storing the data;

[0009] Step 2: Feature extraction: Extract key features from the preprocessed images, including pixels and texture features. Analyze the changes in electrical signals, temperature distribution, and environmental parameters, and extract feature information related to faults;

[0010] Step 3: Fault judgment model establishment: Construct a multi-dimensional feature fault library containing several fault cases, and associate fault phenomena with underlying causes (such as TFT leakage, COF solder joint voids);

[0011] Step 4: Dynamic test mode generation: Dynamically generate targeted test images based on the preliminary detection results to trigger potential faults; Reinforcement learning optimization path: Iteratively optimize the test sequence through the Q-learning algorithm to shorten the detection time while ensuring coverage;

[0012] Step 5: Fault diagnosis: Collect the detection results of the display screen for the optimized test image sequence, comprehensively evaluate whether there are faults in the display screen, as well as the type and severity of the faults, and obtain a fault diagnosis report for the display screen.

[0013] Optionally, the multi-source signal synchronous acquisition method described in Step 1 is as follows:

[0014] S1: Connect the optical sensor to the data acquisition host through a data cable. The installation position of the optical sensor covers the visible area of the display screen and avoids occlusion and reflection. Connect the electrical signal probe to the node of the driving circuit of the display screen to ensure good electrical contact. The probe is connected to the data acquisition host through a shielded cable to reduce external electromagnetic interference; Adjust the focal length and position of the thermal imaging module so that it focuses on the surface of the display screen. Transmit the thermal imaging data to the data acquisition host through an Ethernet interface. The environmental sensor is installed in the area where the display is located and communicates with the data acquisition host through an interface. Create a main thread to coordinate the data acquisition tasks of each sensor, and create independent sub-threads for each sensor;

[0015] S2: Create a main thread to coordinate the data acquisition tasks of each sensor, and create independent sub-threads for each sensor;

[0016] S3: For the optical sensor thread, when the main thread receives the start acquisition instruction, this sub-thread starts to obtain image data from the camera at the set frame rate. In the electrical signal probe sub-thread, the voltage and current data transmitted by the probe are read through a loop and stored in a preset buffer. The thermal imaging module sub-thread obtains thermal imaging data at the set time interval, and the environmental sensor sub-thread continuously collects ambient light and humidity data and updates the corresponding variables;

[0017] S4: Define the storage format for each type of sensor data.

[0018] Optionally, the feature extraction method described in step two is as follows:

[0019] S1: Optical feature extraction: Extract pixel statistical features and calculate statistical parameters such as the mean, variance, and standard deviation of the image;

[0020] S2: Electrical signal feature extraction: Extract time-domain features and calculate the effective value, average value, and peak value of the voltage and current signals;

[0021] S3: Thermal imaging feature extraction: Extract temperature statistical features, calculate the average temperature, maximum temperature, and minimum temperature of the thermal imaging image. The average temperature is obtained by adding up the temperature values of all pixel points and dividing by the total number of pixels, and analyze the features of the temperature histogram;

[0022] S4: Obtain the light intensity value measured by the ambient light sensor and the humidity value measured by the humidity sensor as environmental features.

[0023] Optionally, the construction method of the multi-dimensional feature fault library described in step three is as follows: Integrate the data collected from each sensor and extracted features to form a data set containing multiple feature vectors. For each set of collected data, combine its corresponding image feature vector, electrical signal feature vector, thermal imaging feature vector, and environmental feature vector; Label the collected data, and the labeling content includes whether there is a fault and the type of the fault; And select a database management system to store the multi-dimensional fault feature library, design the database table structure, including data tables, fault type tables, and sensor information tables. The data table is used to store the collected data and the corresponding feature vectors, the fault type table records the information of different fault types, and the sensor information table stores the relevant parameters and status information of the sensors.

[0024] Optionally, the multi-source signal synchronous acquisition and preprocessing described in step one includes a timing correlation module, and the timing correlation module includes a hardware clock synchronization unit, a software timestamp calibration unit, and an association rule mining unit;

[0025] Hardware Clock Synchronization Unit: In the multi-source data acquisition module, a hardware clock synchronization device is adopted to synchronize the device clocks of each sensor with the standard time server through the NTP protocol; and a timer interrupt program is set in the multi-source data acquisition module. Whenever the timer is triggered, a synchronization signal is sent to each sensor simultaneously, so that each sensor records data and stamps timestamps at the same moment;

[0026] Software Timestamp Calibration Unit: In the multi-source data acquisition module, for each collected data sample, its acquisition timestamp is recorded, and the data acquisition timestamps of different sensors are calibrated to have the same frequency;

[0027] Association Rule Mining Unit: The Apriori algorithm is used to discover the association rules between sensor data. First, the data set is scanned to find all frequently occurring single-item sensor data features, and then the frequently occurring multi-item sensor data combination features are gradually found through an iterative method, and finally association rules are generated.

[0028] Optionally, the algorithm implementation steps of the Apriori algorithm are as follows:

[0029] S1: Data Acquisition and Integration: Collect data from optical sensors, electrical signal probes, thermal imaging modules, and environmental sensors;

[0030] S2: Transaction Database Construction: Each set of preprocessed data is regarded as a transaction;

[0031] S3: Frequent Item Set Mining: It includes frequent item set mining of single-item sensor data features and frequent item set mining of multi-item sensor data combination features;

[0032] S4: Association Rule Generation: For each frequent item set, all possible non-empty subset combinations are generated as the antecedent and consequent of the rule, rules are generated, and the confidence of each rule is calculated; if the confidence meets the given minimum confidence threshold, the rule is retained;

[0033] S5: Rule Optimization: Remove redundant rules.

[0034] Optionally, the dynamic test mode generation in step four includes a preliminary detection module, a dynamic test image generation module, and a reinforcement learning optimization module;

[0035] Preliminary Detection Module: Use an optical sensor to collect the initial image of the display screen, preprocess the image, and then preliminarily judge the area where a fault may exist through a feature extraction algorithm;

[0036] Dynamic test image generation module: According to the preliminary detection results, targeted test images are generated through a deep neural network. This module contains multiple sub-networks for generating different types of test images, including a high-frequency flicker image generation sub-network, a gradient gray-scale image generation sub-network, and a polar coordinate grid image generation sub-network. Each sub-network receives the output results of the preliminary detection module and then generates the corresponding test images; Reinforcement learning optimization module: The Q-learning algorithm is used to optimize the generated sequence of test images. This module records the execution effects of each test image (evaluated by reward values) and adjusts the order and selection of the test sequence based on these records to shorten the detection time while ensuring coverage.

[0037] Optionally, the training process of the high-frequency flicker image generation sub-network is as follows: Prepare a number of labeled display screen image data, and the annotation content includes the location and type of the fault area. 70% of the data is used as the training set, and 30% of the data is used as the validation set; For each training sample, the labeled fault area information is input into the neural network, and the difference between the output high-frequency flicker image and the actual labeled flicker image is calculated. The weights and biases of the neural network are adjusted through the backpropagation algorithm to minimize the difference. The performance of the trained neural network is verified on the validation set, and the hyperparameters are adjusted according to the verification results.

[0038] The training process of the gradient gray-scale image generation sub-network is as follows: Collect a number of display screen images of different fault types as training data, extract the fault area information for each image and label the corresponding gradient gray-scale image, and use the supervised learning method to train the neural network to make the output gradient gray-scale image as close as possible to the labeled image. The cross-entropy loss function is used to measure the difference and adjust the model parameters during the training process.

[0039] The training process of the polar coordinate grid image generation sub-network is as follows: Construct a training data set, which contains fault area information and corresponding polar coordinate grid image annotations. The input data is serialized and then input into the RNN for training. During the training process, the performance of the model is evaluated by calculating the accuracy between the predicted polar coordinate grid image and the labeled image, and the model parameters are adjusted according to the loss value.

[0040] Optionally, the way the Q-learning algorithm optimizes the generated sequence of test images is as follows:

[0041] S1: State definition: The state space S consists of the following elements: The state information of the currently to-be-detected display screen area, including the results of the detected areas and the characteristics of the remaining undetected areas; The historical record of the selected test image sequence, including the type of test image, the execution order, and the detection results of each test image in this state.

[0042] S2: Action Definition: The action space A includes selecting the next test image to be executed. After each action selection, the corresponding test image is executed and its effect is observed;

[0043] S3: Reward Function Design: The reward function R is determined based on the change in state after the execution of an action. If a new fault area is detected in the test image, a positive reward is given. If the test image misjudges a fault area, a negative reward is given;

[0044] S4: Q - Table Update: Initialize the Q - table, set the Q - values of all state - action pairs to 0 or a small random value, and update the Q - table according to the actual reward and the estimated future reward after the execution of an action;

[0045] S5: Policy Improvement: Select the optimal policy according to the updated Q - table. In each state, select the action with the highest Q - value as the optimal action. During the training process, continuously iterate and execute the above steps until the Q - table converges or reaches the preset number of training rounds.

[0046] An LCD display fault judgment system, including a multi - source data acquisition module, a central processor, a memory, and a user interface;

[0047] The multi - source data acquisition module includes an optical sensor, an electrical signal probe, a thermal imaging module, and an environmental sensor and their data acquisition host; The optical sensor is a camera, which is used to collect the image data of the LCD display, obtain the display content, brightness, and color information of the screen, so as to analyze whether there are pixel point faults, line faults, and color block abnormal display problems; The electrical signal probe is a voltage - current probe, which monitors the driving voltage and current of the LCD display, and can obtain the waveform, amplitude, and frequency parameters of the electrical signal in real time, so as to judge whether the power supply circuit is normal and whether the signal transmission is stable, and helps to discover display abnormalities caused by circuit faults; The thermal imaging module is a thermal imager, which detects the temperature distribution of each area of the LCD display through thermal imaging technology, and can timely discover potential faults caused by overheating or poor heat dissipation; The environmental sensor is a light sensor and a humidity sensor; It collects ambient light intensity and humidity environment data;

[0048] Central Processor: Responsible for coordinating the work of each sensor, receiving the data transmitted by the sensors, and performing preliminary processing and analysis, running the fault judgment algorithm, and comparing and judging the data according to the preset rules and models to determine whether there is a fault in the LCD display, as well as the type and severity of the fault;

[0049] Memory: It is used to store system programs, a fault feature library, and sensor calibration parameter data. The system program is a software instruction set that controls the operation of the entire system; the fault feature library contains feature information of various common faults and corresponding handling measures; the sensor calibration parameters are used to ensure the accuracy and reliability of sensor measurements.

[0050] User Interface: It is used to set system parameters, start and stop data collection and fault judgment processes, view fault judgment results and historical records, realize the automatic control and operation management of the system, and automatically control the working states and data collection processes of each hardware component according to preset conditions and rules.

[0051] The present invention provides a method and system for judging LCD display screen faults, having the following beneficial effects:

[0052] 1. The present invention synchronously collects a variety of data through an optical sensor, an electrical signal probe, a thermal imaging module, and an environmental sensor, including display data, drive voltage / current monitoring data, temperature distribution, and environmental parameters, etc., and can obtain all-round information of the LCD display screen during operation, avoiding the limitations and misjudgments that may be brought by a single data source; preprocessing operations such as denoising, adjusting contrast and brightness on the collected data can effectively improve the quality of the data, reduce noise interference, make the data more accurate and clear, and are beneficial to subsequent feature extraction and fault judgment.

[0053] 2. The present invention constructs a multi-dimensional feature fault library containing several fault cases and associates the fault phenomena with the underlying causes, providing a valuable reference basis for fault judgment. In actual fault diagnosis, the type and cause of the fault can be quickly and accurately judged by comparing and analyzing with the cases in the fault library, improving the efficiency and accuracy of fault diagnosis.

[0054] 3. The present invention dynamically generates targeted test images according to the preliminary detection results, which can stimulate potential faults and make potential problems more likely to be exposed. This method of dynamically generating test images can be adjusted and optimized according to the actual situation, improving the effectiveness and coverage rate of the test; iteratively optimizing the test sequence through the Q-learning algorithm, shortening the detection time while ensuring the coverage rate, and the reinforcement learning algorithm can automatically adjust the strategy according to the environmental feedback. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic flow chart of the method for judging LCD display screen faults of the present invention;

[0056] Figure 2 It is a schematic module diagram of the system for judging LCD display screen faults of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0057] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0058] Embodiment 1

[0059] Please refer to Figure 1 , a method for judging display faults of an LCD display screen, comprising the following steps:

[0060] Step 1: Multi-source signal synchronous acquisition and preprocessing: Use an optical sensor (high-frame-rate camera), an electrical signal probe, a thermal imaging module, and an environmental sensor (such as ambient light, ambient humidity) to synchronously acquire the display data, drive voltage / current monitoring data, temperature distribution, and environmental parameters of the LCD display screen, and preprocess the acquired data, including denoising, adjusting contrast and brightness, ensuring the quality and consistency of the image, and storing the data;

[0061] Step 2: Feature extraction: Extract key features from the preprocessed image, including pixels and texture features, analyze the changes in electrical signals, temperature distribution, and environmental parameters, and extract feature information related to faults;

[0062] Step 3: Fault judgment model establishment: Construct a multi-dimensional feature fault library containing several fault cases, and associate fault phenomena with underlying causes (such as TFT leakage, COF virtual soldering);

[0063] Step 4: Dynamic test mode generation: According to the preliminary detection results, dynamically generate targeted test images (such as high-frequency flicker, gradient gray scale, polar coordinate grid) to stimulate potential faults (such as liquid crystal response delay, pixel adhesion). For example, for an area where liquid crystal response delay may occur, generate a high-frequency flicker test image, observe the display situation of this area at different flicker frequencies, and thus judge whether there is a liquid crystal response delay fault;

[0064] Reinforcement learning optimization path: Iteratively optimize the test sequence through the Q-learning algorithm, while ensuring coverage, shorten the detection time (reduce redundant test steps by more than 30%). For example, in the initial test stage, the system randomly selects some test images for detection, calculates the reward value of each test image according to the detection results, and then continuously adjusts the test sequence through the Q-learning algorithm, preferentially selecting test images with high reward values, thereby improving the detection efficiency;

[0065] Step 5: Fault Diagnosis: Collect the detection results of the optimized test image sequence for the display screen, comprehensively evaluate whether there are faults in the display screen, as well as the type and severity of the faults, and obtain a fault diagnosis report for the display screen. For example, by comparing the response of the display screen under different test images, judge whether there are liquid crystal response delays and pixel adhesion faults.

[0066] The multi-source signal synchronous acquisition method in Step 1 is as follows:

[0067] S1: Connect the optical sensor to the data acquisition host through a data cable (such as USB3.0). The installation position of the optical sensor covers the visible area of the display screen and avoids occlusion and reflection. Connect the electrical signal probe to the node of the driving circuit of the display screen to ensure good electrical contact. The probe is connected to the data acquisition host through a shielded cable to reduce external electromagnetic interference; adjust the focal length and position of the thermal imaging module so that it focuses on the surface of the display screen, and transmit the thermal imaging data to the data acquisition host through the Ethernet interface. The environmental sensor is installed in the area where the display is located and communicates with the data acquisition host through the interface. Create a main thread to coordinate the data acquisition tasks of each sensor, and create an independent sub-thread for each sensor;

[0068] S2: Create a main thread to coordinate the data acquisition tasks of each sensor, and create an independent sub-thread for each sensor;

[0069] S3: For the optical sensor thread, when the main thread receives the start acquisition instruction, this sub-thread starts to obtain image data from the camera at the set frame rate. In the electrical signal probe sub-thread, the voltage and current data transmitted by the probe are read through a loop and stored in a preset buffer. The thermal imaging module sub-thread obtains thermal imaging data at the set time interval (such as frames per second, specific time interval value). The environmental sensor sub-thread continuously collects ambient light and humidity data and updates the corresponding variables;

[0070] S4: Define the storage format for each type of sensor data. For example, image data is stored in JPEG format and the file is named according to the acquisition time. The voltage and current data are stored as text files or binary files, and each line records the relevant data at a time point, including the timestamp, voltage value, and current value.

[0071] The method of feature extraction is as follows:

[0072] S1: Optical feature extraction: Extract pixel statistical features, and calculate statistical parameters such as the mean, variance, and standard deviation of the image. For example, for a grayscale image, the mean reflects the average brightness of the image, and the calculation formula is: where M and N are the number of rows and columns of the image respectively, and P(i,j) represents the gray value of the pixel point (i,j);

[0073] Extract the histogram features of the image, including the entropy, energy, and contrast of the histogram. Entropy is used to measure the information richness of the image, and the calculation formula is: where p(k) is the probability that a pixel with a gray value of k appears, and L is the total number of gray levels;

[0074] Extract the texture features of the image. Use the gray-level co-occurrence matrix (GLCM) to extract texture features. Calculate the GLCM in different directions (such as 0°, 45°, 90°, 135°), and extract the contrast, correlation, energy, and uniformity features from it. For example, the contrast feature reflects the sharpness of the image texture, and the calculation formula is: Contrast = ∑ i ∑ j (i - j) 2 P(i, j), where P(i, j) is the value of the element (i, j) in the GLCM;

[0075] Use the local binary pattern (LBP) operator to extract texture features. The LBP operator generates a binary coding sequence to describe the local texture structure of the image by comparing the gray value relationship between the central pixel and its surrounding pixels;

[0076] S2: Electrical signal feature extraction: Extract time-domain features, calculate the effective value, average value, and peak value of the voltage and current signals. The effective value is calculated by the formula: where x i is the voltage or current value of the sampling point, and N is the number of sampling points;

[0077] Extract the pulse width, rise time, and fall time parameters of the signal. Determine these time parameters by detecting the change of the signal edge. For example, the rise time is defined as the time required for the signal to rise from the low level to a certain threshold level;

[0078] Extract frequency-domain features. Perform a fast Fourier transform (FFT) on the voltage and current signals to obtain the frequency spectrum distribution of the signal. Extract the frequency components, amplitudes, and phase information from the frequency spectrum. For example, calculate the frequency values and corresponding amplitudes of the main frequency components, as well as the energy proportion of each frequency component;

[0079] S3: Thermal imaging feature extraction: Extract temperature statistical features, calculate the average temperature, maximum temperature, and minimum temperature of the thermal imaging image. The average temperature is obtained by adding the temperature values of all pixel points and then dividing by the total number of pixels. Analyze the features of the temperature histogram, such as the skewness and kurtosis of the temperature distribution. The skewness reflects the degree of asymmetry of the temperature distribution, and the calculation formula is: where T i is the temperature value of the i-th pixel point, μ is the average temperature, and σ is the standard deviation of the temperature;

[0080] Extract the features of the hot spot area, and detect the hot spot area in the thermal imaging image by setting a temperature threshold. For example, mark the area with a temperature higher than a certain threshold as the hot spot area, and calculate the area, perimeter, and position geometric features of the hot spot area;

[0081] S4: Obtain the light intensity value measured by the ambient light sensor and the humidity value measured by the humidity sensor as environmental features.

[0082] The multi-dimensional feature fault library in Step 3 is constructed as follows: Integrate the data collected and extracted from each sensor to form a data set containing multiple feature vectors. For example, for each set of collected data, combine its corresponding image feature vector, electrical signal feature vector, thermal imaging feature vector, and environmental feature vector; Invite professional display fault diagnosis experts to label the collected data, and the labeling content includes whether there is a fault and the type of the fault (such as abnormal LCD screen display, backlight fault, drive circuit fault); And select a database management system (such as MySQL, Oracle) to store the multi-dimensional fault feature library, and design the database table structure, including data tables, fault type tables, and sensor information tables. The data table is used to store the collected data and the corresponding feature vectors, the fault type table records the information of different fault types, and the sensor information table stores the relevant parameters and status information of the sensors.

[0083] Integrate the data collected and extracted from each sensor to form a data set containing multiple feature vectors such as images, electrical signals, thermal imaging, and environment. This method can obtain the status information of the LCD display from different angles. For example, the image feature vector can reflect visual abnormalities such as pixels and textures on the screen; the electrical signal feature vector can reflect circuit-related problems such as driving voltage and current; the thermal imaging feature vector can show temperature distribution abnormalities in different areas of the screen, while the environmental feature vector records the impact of environmental factors on the screen. Combining these feature vectors provides a basis for fault diagnosis from multiple dimensions, greatly enriching the information source for judging faults and making the judgment of display faults more comprehensive and accurate;

[0084] Invite professional display fault diagnosis experts to label the collected data. With their profound professional knowledge and rich practical experience, experts can accurately judge whether there is a fault and the type of the fault, such as abnormal LCD screen display, backlight fault, drive circuit fault, etc. This professional labeling assigns accurate labels to the data in the multi-dimensional feature fault library, providing a reliable reference standard for subsequent analysis and fault judgment based on these data, and improving the accuracy and credibility of the entire fault diagnosis method;

[0085] Use a database management system to store the multi-dimensional fault feature library. These database systems have good data storage and management capabilities. They can efficiently store a large amount of feature vector data, annotation information, and sensor-related information, and can quickly retrieve the required data according to the user's needs. During the fault diagnosis process, when querying specific types of faults or a certain group of sensor data, it can be quickly extracted from the database, providing strong support for timely and accurate fault judgment.

[0086] In this embodiment, a single sensor or data source may only provide limited information about the state of the LCD display screen, while multi-modal data can capture information from different aspects. For example, an optical sensor can detect image quality, an electrical signal probe can monitor electrical parameters such as voltage and current, a thermal imaging module can reflect the temperature distribution, and an environmental sensor can provide the impact of environmental conditions on the display screen. Fusing these different modal data can provide a more comprehensive understanding of the working state of the LCD display screen, reducing the possibility of missed detections and misjudgments. Different modal data can describe and analyze the LCD display screen from multiple angles. Through a reasonable fusion algorithm, they can verify and complement each other, thereby improving the accuracy of fault judgment.

[0087] The dynamic fusion of multi-modal data can process the data collected by different sensors in real time, update the assessment of the state of the LCD display screen in a timely manner, and achieve fast and accurate fault judgment. For some occasions with high real-time requirements, such as the detection of LCD display screens on industrial automation production lines, it can detect and handle faults in a timely manner, avoiding the production of a large number of defective products.

[0088] Different modal data describe and analyze the display screen from multiple angles, corroborating and complementing each other. For example, for a suspected defective area found by an optical sensor, an electrical signal probe can be used to detect whether the circuit in this area is normal, and a thermal imaging module can be used to check whether there is overheating in this area, so as to more accurately judge whether there is a real defect in this area. The dynamic fusion detection technology of multi-modal data shows significant advantages and benefits in the patent of the LCD display screen picture quality detection system, not only improving the comprehensiveness, accuracy, and efficiency of detection.

[0089] Embodiment 2

[0090] In this embodiment, the following optimizations are made on the basis of Embodiment 1. Specifically, in the multi-source signal synchronous acquisition and preprocessing in Step 1, it includes a timing correlation module, and the timing correlation module includes a hardware clock synchronization unit, a software timestamp calibration unit, and an association rule mining unit.

[0091] Hardware Clock Synchronization Unit: In the multi-source data acquisition module, a hardware clock synchronization device is adopted, such as a Network Time Protocol (NTP) server or a precision clock source (such as a GPS clock). Through the NTP protocol, the device clocks of each sensor are synchronized with the standard time server to ensure the accuracy and consistency of timestamps. A timer interrupt program is set in the multi-source data acquisition module. Whenever the timer is triggered, a synchronization signal is sent to each sensor simultaneously, enabling each sensor to record data and attach timestamps at the same moment.

[0092] Software Timestamp Calibration Unit: In the multi-source data acquisition module, for each collected data sample, its acquisition timestamp is recorded, and the data acquisition timestamps of different sensors are calibrated to have the same frequency. For example, assume the frame rate of the optical sensor is 30fps, the sampling rate of the electrical signal probe is 1kHz, and the frame rate of the thermal imaging module is 10fps. When a set of data is collected, find the smallest time unit (such as milliseconds) among them, and uniformly convert the data timestamps of each sensor into multiples of this unit. If the timestamp of a frame of image collected by the optical sensor is t1, and the electrical signal probe collects multiple data points at the moment of t1, select the data point closest to t1 as the timestamp data of the electrical signal corresponding to the optical image. For the existing time delay error, interpolation or curve fitting methods are used for correction. For example, by analyzing the data transmission delay law between sensors, a function relationship model between the delay time and sensor data is established to correct the timestamp.

[0093] Association Rule Mining Unit: The Apriori algorithm is used to discover the association rules between sensor data. First, scan the data set to find all single-item sensor data features that frequently appear (i.e., features that meet the minimum support threshold), and then gradually find the multi-item sensor data combination features that frequently appear through an iterative method, and finally generate association rules.

[0094] The algorithm implementation steps of the Apriori algorithm are as follows:

[0095] S1: Data Acquisition and Integration: Collect data from the optical sensor, electrical signal probe, thermal imaging module, and environmental sensor. Assume that n groups of data are collected, and each group of data contains optical sensor data O i (such as image feature vectors), electrical signal probe data E i (voltage and current values), thermal imaging data T i (temperature distribution information), and environmental sensor data H i(Ambient light, humidity values), preprocess the collected data, including data cleaning (removing outliers and noise), standardization (converting data of different modalities to a unified dimension or scale), and discretization (converting continuous data into discrete intervals or categories). For example, for the voltage data of the electrical signal probe, it is divided into three intervals: low, medium, and high according to its value range;

[0096] S2: Transaction database construction: Treat each set of preprocessed data as a transaction. For example, the j-th transaction is represented as T = {O j ,E j ,T j ,H j}, where, O j is the presence or absence of certain features in the image (such as whether there is a bright spot defect feature), E j is the set of electrical signal intervals, T j is the set of temperature intervals, H j is the set of environmental parameter intervals. In this way, all transactions form the transaction database D;

[0097] S3: Frequent itemset mining: Includes frequent itemset mining of single-sensor data features and frequent itemset mining of combined features of multiple-sensor data;

[0098] Frequent itemset mining of single-sensor data features (L1): Set the minimum support threshold as min-sup. For the data features of each sensor modality, count the number of times it appears in the dataset. For example, for the feature o of the optical sensor, calculate its frequency count(o) in all transactions. If then add the feature o to the frequent 1-itemset L1. Perform the same operation on the features of the electrical signal probe, thermal imaging, and environmental sensors respectively to obtain their respective initial frequent 1-itemsets;

[0099] Frequent itemset mining of combined features of multiple-sensor data (L k , k≥2): Assume that the frequent (k - 1)-itemset L k -1 has been obtained. Generate the set C k of candidate k-itemsets through a self-join operation. For example, for two frequent (k - 1)-itemsets l1 and l2, if their first (k - 2) features are the same, then merge them to generate a candidate k-itemset. For each candidate k-itemset c, calculate its support: where, count(c) is the number of transactions containing c. If Support(c) ≥ min-sup, then add c to the frequent k-itemset L k ; Repeat the above steps until no new frequent itemsets can be generated, that is, obtain all frequent itemsets L = L1 ∪ L2 ∪ … ∪ Lm ;

[0100] S4: Association rule generation: For each frequent itemset l ∈ L (assuming l contains k features), generate all possible non-empty subset combinations as the antecedent (Left Hand Side, LHS) and consequent (Right Hand Side, RHS) of the rule. For example, for the frequent 2-itemset {o, e} (where o is the optical sensor feature and e is the electrical signal probe feature), generate the rules o → e and e → o, and calculate the confidence of each rule: If the confidence meets the given minimum confidence threshold min_conf, then retain the rule;

[0101] S5: Rule optimization: Remove redundant rules. If there are two rules R1: A → B and R2: A → C, and the confidences of the two rules are similar at the same time, then remove rule R2 because R1 can already cover some cases of R2. Adjust the rules according to expert experience or actual application scenarios. For example, in some specific display working environments, some rules may not be applicable and need to be corrected or deleted;

[0102] Also consider the timeliness and order of the data:

[0103] Time window partitioning: According to actual needs and data characteristics, select an appropriate time window size. For example, divide the time window in units of every 10 seconds or every minute. Within each time window, only perform association analysis on the sensor data falling within that window to ensure that the association rules are mined based on the data relationships within the same time period and are more in line with the actual temporal logic;

[0104] Sequential data processing: When performing frequent itemset mining, not only consider whether each feature appears simultaneously, but also consider the order in which they appear. This is achieved by adding order labels to the features in each transaction. For example, for a transaction T = {O1, E2, T3, H4} (where the numbers represent the order in which the features appear), when generating frequent itemsets and association rules, give priority to the order factor. For example, the rules O1 → E2 (indicating the probability that when the feature 1 of the optical sensor appears first, the feature 2 of the electrical signal probe also appears) and E2 → O1 (indicating the probability that when the feature 2 of the electrical signal probe appears first, the feature 1 of the optical sensor also appears) may have different confidences and supports, and need to be calculated and evaluated separately);

[0105] Rule evaluation and screening: Evaluate the mined association rules, calculate their support and confidence metrics, remove those rules with too low support or confidence, and retain strong association rules as the basis for fault judgment. According to the actual application scenario and expert experience, screen and optimize the association rules. For example, some association rules may only apply to specific display models or working environments and need to be adjusted and improved according to the specific situation.

[0106] In this embodiment, by using a Network Time Protocol (NTP) server or a precision clock source (such as a GPS clock) and synchronizing the device clocks of each sensor with the standard time server using the NTP protocol, it is possible to ensure that the timestamps from different sensors are accurate and consistent. This is crucial for subsequent fault analysis because accurate time information can precisely locate the moment when a fault occurs and avoid misjudgment of the cause of the fault due to inaccurate time. Set a timer interrupt program. Whenever the timer is triggered, a synchronization signal is sent to each sensor simultaneously, causing each sensor to record data and attach timestamps at the same moment. This further enhances the synchronization of data acquisition, especially in cases where real-time correlation analysis of data from multiple sensors is required.

[0107] For sensors with different sampling rates and frame rates, such as optical sensors (30fps), electrical signal probes (1kHz), and thermal imaging modules (10fps), uniformly convert their data timestamps into multiples of the smallest time unit (such as milliseconds) so that data from different sensors can be analyzed on the same time scale. This helps integrate multi-source data and discover potential relationships between the data. Use interpolation or curve fitting methods to correct the existing time delay errors, consider the data transmission delay law between sensors, and establish a function relationship model between the delay time and sensor data to correct the timestamps. This can improve the reliability and accuracy of the data and avoid data misalignment caused by time delay.

[0108] Use the Apriori algorithm to discover association rules between sensor data, which can mine valuable information hidden in the data. By scanning the data set to find frequently occurring single-item sensor data features, and then gradually finding multi-item sensor data combination features, and finally generating association rules. This helps understand the mutual relationship between different sensor data and provides new ideas for fault judgment. In the fault judgment of LCD display screens, through association rule mining, it may be found that when the optical sensor detects abnormal brightness in a specific area, at the same time the electrical signal probe detects that the voltage fluctuation in this area exceeds a certain range, and the thermal imaging module shows that the temperature in this area rises, there is a high probability of a bad pixel fault in the display screen. Such association rules can help technicians quickly locate the cause of the fault and improve the efficiency of fault troubleshooting.

[0109] Embodiment 3

[0110] This embodiment makes the following optimizations based on Example 1 or Example 2. Specifically, the generation of the dynamic test mode in Step 4 includes a preliminary detection module, a dynamic test image generation module, and a reinforcement learning optimization module;

[0111] Preliminary detection module: Use an optical sensor to collect the initial image of the display screen, preprocess the image (such as noise reduction, contrast enhancement), and then preliminarily judge the areas that may have faults through feature extraction algorithms (such as edge detection, pixel value statistical analysis). For example, through the edge detection algorithm, check whether there are obvious bright or dark abnormal areas in the display screen;

[0112] Dynamic test image generation module: According to the preliminary detection results, generate targeted test images through a deep neural network. This module contains multiple sub-networks for generating different types of test images, including a high-frequency flicker image generation sub-network, a gradient gray-scale image generation sub-network, and a polar coordinate grid image generation sub-network. Each sub-network receives the output results of the preliminary detection module, including the location and type information of the possible fault areas, and then generates the corresponding test images;

[0113] Reinforcement learning optimization module: Use the Q-learning algorithm to optimize the generated test image sequence. This module records the execution effect of each test image (evaluated by the reward value), and adjusts the order and selection of the test sequence according to these records to shorten the detection time while ensuring the coverage rate;

[0114] Data interaction process:

[0115] S1: The preliminary detection module transmits the processed preliminary detection results (possible fault area information) to the dynamic test image generation module;

[0116] S2: The dynamic test image generation module generates an initial set of test image sequences based on the received information, provides these test images to the display screen for testing, and transmits the information during the test (such as test image type, display effect, etc.) to the reinforcement learning optimization module;

[0117] S3: The reinforcement learning optimization module calculates the reward value of each test image based on the received information, updates the test sequence through the Q-learning algorithm, and feeds back the optimized test sequence to the dynamic test image generation module for guiding the generation of the next round of test images;

[0118] The neural network structure of the high-frequency flicker image generation sub-network includes:

[0119] Input layer: Receives information about the possible locations and types of fault areas, divides the display screen into m×n areas, and uses an m×n-dimensional vector to represent whether there is a fault suspicion in each area (1 means yes, 0 means no);

[0120] Hidden layer: Adopts a multi-layer perceptron (MLP) structure, contains h1 neurons, and the activation function is ReLU. The output of the input layer is passed through a fully connected layer to the hidden layer, and the calculation formula is: z (1) =W (1) x + b (1) , a (1) =ReLU(z (1) ); where W (1) is the weight matrix from the input layer to the hidden layer, b (1) is the bias vector, and x is the input vector;

[0121] Output layer: Outputs the pixel value matrix of the high-frequency flickering image. For each pixel point (i, j), the output layer calculates its grayscale value y ij : where is the weight matrix from the hidden layer to the output layer, is the bias vector, and σ is the sigmoid activation function, which is used to map the pixel value to between 0 and 1;

[0122] The training process of the high-frequency flickering image generation sub-network is as follows:

[0123] Prepare a number of display screen image data with annotations. The annotation content includes the location and type of the fault area. Take 70% of the data as the training set and 30% of the data as the validation set; for each training sample, input the annotated fault area information into the neural network, calculate the difference (such as the mean square error) between the output high-frequency flickering image and the actual annotated flickering image, and adjust the weights and biases of the neural network through the backpropagation algorithm to minimize the difference. Verify the performance of the trained neural network on the validation set, and adjust the hyperparameters (such as the learning rate, the number of neurons in the hidden layer) according to the verification results; for some faults that are difficult to detect under normal display but are easy to expose under specific frequency flickering conditions, such as some poor line contacts, small pixel point abnormalities, etc., the high-frequency flickering image can make them easier to be discovered, because such faults may not be obvious under static or conventional dynamic display, while high-frequency flickering can stimulate the fault characteristics and enable the detector or system to more accurately capture the fault signs;

[0124] The neural network structure of the gradient gray-scale image generation sub-network includes:

[0125] Input layer: Receives fault area information, divides the display screen into m×n areas, and uses an m×n-dimensional vector to represent whether there is a fault suspicion in each area (1 means yes, 0 means no);

[0126] Hidden layer: Adopting the structure of Convolutional Neural Network (CNN), it contains k convolutional kernels, each with a size of c×c. The convolution operation extracts features from the input data, and the calculation formula is: where W is the convolutional kernel weight matrix, x is the input data (fault area information), and b is the bias vector. represents the convolution operation;

[0127] Output layer: Outputs the grayscale matrix of the gradient grayscale image. For each pixel point (i,j), the output layer calculates its grayscale value: y ij = W' ij a + b' ij where W' ij is the weight matrix from the hidden layer to the output layer, and b' ij is the bias vector;

[0128] The training process of the gradient grayscale image generation sub-network is as follows:

[0129] Collect a number of display screen images of different fault types as training data. Extract the fault area information from each image and label the corresponding gradient grayscale image. Use the supervised learning method to train the neural network to make the output gradient grayscale image as close as possible to the labeled image. During the training process, use the cross-entropy loss function to measure the difference and adjust the model parameters.

[0130] The grayscale display ability of the LCD display screen is one of its important performance indicators. The gradient grayscale image can help detect the performance of the display screen during the transition of different brightness levels. If there are faults, such as abnormal liquid crystal arrangement, drive circuit faults, etc., unevenness, stripes, color blocks and other abnormal phenomena will appear during the grayscale transition. Through this targeted test image, such faults related to grayscale display can be accurately detected;

[0131] The neural network structure of the polar coordinate grid image generation sub-network includes:

[0132] Input layer: Receives the position and size information of the fault area. For example, the center coordinates (x,y) and radius r of each fault area are represented by a four-dimensional vector;

[0133] Hidden layer: Adopting the structure of Recurrent Neural Network (RNN), it contains l neurons. The RNN processes sequential data, generates a sequence of polar coordinate grid images according to the input fault area information, and the output of the input layer is passed to the hidden layer according to the time step. The state update formula of the hidden layer is: h t = σ(Wh t-1 + U xt + b), where h tis the state of the hidden layer at time t, W is the weight matrix of the hidden layer, U is the weight matrix input to the hidden layer, xt is the input vector at time t, b is the bias vector, and σ is the activation function;

[0134] Output layer: Convert the state of the hidden layer into a pixel matrix of the polar coordinate grid image. For each pixel point (i, j), calculate whether its position in polar coordinates is within the fault area according to the state of the hidden layer. If it is, set it to a specific value (such as 1), otherwise set it to another value (such as 0);

[0135] The training process of the polar coordinate grid image generation sub-network is as follows:

[0136] Construct a training dataset, which contains fault area information and corresponding polar coordinate grid image annotations. Serialize the input data and input it into the RNN for training. During the training process, evaluate the model performance by calculating the accuracy between the predicted polar coordinate grid image and the annotated image, and adjust the model parameters according to the loss value;

[0137] This kind of image helps to detect problems such as geometric deformation and pixel arrangement regularity of the display screen. For example, if the display screen is squeezed by external force resulting in local deformation, or the internal pixel structure is misaligned, etc., it will be manifested as abnormalities such as grid distortion and deformation in the polar coordinate grid image, which is convenient for inspectors to intuitively discover such faults related to the physical structure and pixel layout of the display screen;

[0138] The way the Q-learning algorithm optimizes the generated test image sequence is as follows:

[0139] S1: State definition: The state space S consists of the following elements: The state information of the current display screen area to be detected, including the results of the detected areas (whether there are faults) and the characteristics of the remaining undetected areas (such as area, position);

[0140] The historical record of the selected test image sequence, including the type, execution order of the test images, and the detection results (reward values) of each test image in this state. For example, divide the display screen into 9 areas, and the first 3 areas have been detected. Among them, the first area has a fault, and the second and third areas are normal; the selected test image sequence is [high-frequency flashing image, gradually changing grayscale image]. The reward value of the high-frequency flashing image in the first area is 5, and the reward value in other areas is 0; the reward value of the gradually changing grayscale image in all areas is -1. Then the current state can be expressed as: S = ([1, 0, 0, *, *, *, *, *, *]), [(high-frequency flashing image, [5, 0, 0])], [(gradually changing grayscale image, [-1, -1, -1])], where * indicates that the area has not been detected;

[0141] S2: Action Definition: The action space A includes selecting the next test image to be executed. Suppose there are n different test images (such as high-frequency flicker, gradient grayscale, polar coordinate grid), then the action set is: A = {a1, a2, …, a n}, and after each action is selected, the corresponding test image is executed and its effect (obtaining a reward value) is observed;

[0142] S3: Reward Function Design: The reward function R(s, a, s') is determined based on the change of the state after the action is executed. If a new fault area is found in the test image, a relatively high positive reward is given. For example, the reward value is 10. If the test image rules out the suspicion of a large area of non-faulty areas, a certain positive reward is given, such as the reward value is 3. If the test image does not provide effective new information (neither finding new faults nor ruling out more non-faulty areas), a relatively small negative reward is given, such as -1. If the test image misjudges the fault area (judging a faulty area as non-faulty, or vice versa), a relatively large negative reward is given, such as -5. The specific form of the reward function is adjusted and optimized according to actual needs. For example: R(s, a, s') = w1 × the number of newly discovered fault areas - w2 × the number of misjudged fault areas + w3 × the area of non-faulty areas ruled out; where w1, w2, and w3 are weight coefficients, determined according to experience or experiments;

[0143] S4: Q-Table Update: Initialize the Q-table, set the Q values of all state-action pairs to 0 or a small random value, and update the Q-table according to the actual reward and the estimated future reward after the action is executed. For each state-action pair (s, a), the target Q value Q'(s, a) can be calculated by the following formula: Q'(s, a) = R(s, a, s') + γmax a ' ∈A Q(s', a'), where γ is the discount factor (0 ≤ γ < 1), used to balance the importance of immediate reward and future reward; then update the actual Q value according to the target Q value: Q(s, a) = Q(s, a) + α[Q'(s, a) - Q(s, a)], where α is the learning rate;

[0144] S5: Policy Improvement: Select the optimal policy according to the updated Q-table. In each state, select the action with the highest Q value as the optimal action. For example, in state S, if max a∈A Q(s, a) = Q(s, a3), then select action a3 as the optimal policy. As the training progresses and the environment changes (such as changes in the display screen fault type, changes in the test image effect), continuously adjust the Q-table and the policy to adapt to the new detection task). During the training process, continuously iterate and execute the above steps until the Q-table converges or reaches the preset number of training rounds.

[0145] In this embodiment, an optical sensor is used to collect the initial image of the display screen, and information on the actual display state of the screen can be directly obtained. Through a feature extraction algorithm, the image is preprocessed and preliminarily analyzed, and areas that may have faults can be quickly screened out. This greatly reduces the scope that needs to be focused on for subsequent detection, avoids comprehensive and undifferentiated complex detection of the entire display screen, and improves the detection efficiency;

[0146] Each sub-network receives the output result of the preliminary detection module and generates a corresponding test image. Multiple different types of sub-networks cooperate with each other to detect the areas that may have faults from different angles. Whether it is a fault in electrical performance (which may be detected by high-frequency flicker), a fault in gray-scale display ability (detected by means of a gradient gray scale), or a fault in physical structure and pixel arrangement (relied on polar coordinate grid detection), all can be covered, thus greatly improving the comprehensiveness and accuracy of the display screen fault detection;

[0147] The Q-learning algorithm is used to optimize the generated sequence of test images. By recording the execution effects of each test image and adjusting the order and selection of the test sequence according to these records, on the premise of ensuring sufficient detection coverage for various possible fault situations, those test images that contribute little to fault judgment or have high repeatability can be adjusted backward or some of them can be eliminated, so that the entire detection process can focus on key and effective test images more quickly, thus shortening the detection time, especially applicable to the situation where a large number of display screens need to be quickly detected on a large-scale production line.

[0148] Please refer to Figure 2 , the present invention also discloses an LCD display screen display fault judgment system, including a multi-source data acquisition module, a central processing unit, a memory, and a user interface;

[0149] The multi-source data acquisition module includes an optical sensor, an electrical signal probe, a thermal imaging module, an environmental sensor, and their data acquisition host; the optical sensor is a camera, which is used to collect image data of the LCD display screen, obtain the display content, brightness, and color information of the screen, so as to analyze whether there are pixel point failures, line failures, and color block abnormal display problems; the electrical signal probe is a voltage and current probe, which monitors the driving voltage and current of the LCD display screen, and can obtain parameters such as the waveform, amplitude, and frequency of the electrical signal in real time, so as to judge whether the power supply circuit is normal and whether the signal transmission is stable, and helps to discover display abnormalities caused by circuit failures; the thermal imaging module is a thermal imager, which detects the temperature distribution of each area of the LCD display screen through thermal imaging technology, and can timely discover potential failure hazards caused by overheating or poor heat dissipation. For example, the problem of uneven backlight or damage caused by too high temperature of the backlight board; the environmental sensor is a light sensor and a humidity sensor; it collects ambient light intensity and humidity environment data. Too strong or too weak ambient light may affect the visibility and contrast effect of the display screen, and changes in humidity may have a corrosive and adverse effect on the internal electronic components of the display screen. These environmental factors may indirectly cause display failures or affect the display effect;

[0150] Central processing unit: As the core control unit of the system, it is responsible for coordinating the work of each sensor, receiving the data transmitted by the sensors, and performing preliminary processing and analysis, running a fault judgment algorithm, comparing and judging the data according to preset rules and models, so as to determine whether there is a fault in the LCD display screen and the type and severity of the fault;

[0151] Memory: It is used to store system programs, fault feature libraries, and sensor calibration parameter data. The system program is a software instruction set that controls the operation of the entire system; the fault feature library contains feature information of various common faults and corresponding treatment measures; the sensor calibration parameters are used to ensure the accuracy and reliability of sensor measurements;

[0152] User interface: It is used to set system parameters, start and stop the data acquisition and fault judgment processes, view the fault judgment results and historical records, realize the automatic control and operation management of the system, and automatically control the working states and data acquisition processes of each hardware component according to preset conditions and rules.

[0153] In summary, the present invention synchronously collects a variety of data through an optical sensor, an electrical signal probe, a thermal imaging module, and an environmental sensor, including display data, drive voltage / current monitoring data, temperature distribution, and environmental parameters, etc., and can obtain all-round information of the LCD display screen during operation. The acquisition of this multi-source data provides a solid foundation for subsequent accurate fault judgment, avoiding the limitations and misjudgments that may be brought by a single data source; preprocessing operations such as denoising, adjusting contrast and brightness on the collected data can effectively improve the quality of the data, reduce noise interference, make the data more accurate and clear, which is beneficial to subsequent feature extraction and fault judgment. At the same time, the storage of the preprocessed data also provides convenient data support for subsequent analysis;

[0154] Construct a multi-dimensional feature fault library containing several fault cases, and associate the fault phenomenon with the underlying cause, providing a valuable reference basis for fault judgment. In actual fault diagnosis, the type and cause of the fault can be quickly and accurately judged by comparing and analyzing with the cases in the fault library, improving the efficiency and accuracy of fault diagnosis;

[0155] Dynamically generate targeted test images according to the preliminary detection results, which can stimulate potential faults and make potential problems more likely to be exposed. This method of dynamically generating test images can be adjusted and optimized according to the actual situation, improving the effectiveness and coverage rate of the test; Iteratively optimize the test sequence through the Q-learning algorithm, shortening the detection time while ensuring the coverage rate. The reinforcement learning algorithm can automatically adjust the strategy according to the environmental feedback, making the test process more intelligent and efficient. By continuously optimizing the test sequence, the display screen can be comprehensively and accurately detected in the shortest time; Collect the detection results of the display screen for the optimized test image sequence, comprehensively evaluate whether there is a fault in the display screen and the type and severity of the fault, and obtain a fault diagnosis report.

[0156] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for judging display faults of an LCD display screen, characterized in that, It includes the following steps: Step 1: Multi-source signal synchronous acquisition and preprocessing: Use an optical sensor, an electrical signal probe, a thermal imaging module, and an environmental sensor to synchronously acquire the display data, drive voltage / current monitoring data, temperature distribution, and environmental parameters of the LCD display screen. Preprocess the acquired data, including denoising, adjusting contrast and brightness, and store the data; Step 2: Feature extraction: Extract key features from the preprocessed images, including pixels and texture features. Analyze the changes in electrical signals, temperature distribution, and environmental parameters, and extract feature information related to faults; Step 3: Fault judgment model establishment: Construct a multi-dimensional feature fault library containing several fault cases, and associate fault phenomena with underlying causes; Step 4: Dynamic test mode generation: Dynamically generate targeted test images according to the preliminary detection results to trigger potential faults; Reinforcement learning optimization path: Iteratively optimize the test sequence through the Q-learning algorithm to shorten the detection time while ensuring coverage; Step 5: Fault diagnosis: Collect the detection results of the display screen for the optimized test image sequence, comprehensively evaluate whether there are faults in the display screen, as well as the type and severity of the faults, and obtain a fault diagnosis report for the display screen.

2. The method for judging the display fault of an LCD display screen according to claim 1, wherein: The multi-source signal synchronous acquisition method described in Step 1 is as follows: S1: Connect the optical sensor to the data acquisition host through a data cable. The installation position of the optical sensor covers the visible area of the display screen. Connect the electrical signal probe to the node of the driving circuit of the display screen, and the probe is connected to the data acquisition host through a shielded cable; Adjust the focal length and position of the thermal imaging module so that it focuses on the surface of the display screen. Transmit the thermal imaging data to the data acquisition host through an Ethernet interface. The environmental sensor is installed in the area where the display is located and communicates with the data acquisition host through an interface. Create a main thread to coordinate the data acquisition tasks of each sensor, and create an independent sub-thread for each sensor; S2: Create a main thread to coordinate the data acquisition tasks of each sensor, and create an independent sub-thread for each sensor; S3: For the optical sensor thread, when the main thread receives the start acquisition instruction, this sub-thread starts to obtain image data from the camera at the set frame rate. In the electrical signal probe sub-thread, the voltage and current data transmitted by the probe are read in a loop and stored in a preset buffer. The thermal imaging module sub-thread obtains thermal imaging data at the set time interval, and the environmental sensor sub-thread continuously collects ambient light and humidity data and updates the corresponding variables; S4: Define the storage format for each type of sensor data.

3. A method for judging display faults of an LCD display screen according to claim 1, characterized in that: The feature extraction method described in Step 2 is as follows: S1: Optical feature extraction: Extract pixel statistical features, and calculate statistical parameters such as the mean, variance, and standard deviation of the image; S2: Electrical signal feature extraction: Extract time-domain features, and calculate the effective value, average value, and peak value of the voltage and current signals; S3: Thermal imaging feature extraction: Extract temperature statistical features, and calculate the average temperature, maximum temperature, and minimum temperature of the thermal imaging image. The average temperature is obtained by adding the temperature values of all pixel points and dividing by the total number of pixels. Analyze the features of the temperature histogram; S4: Obtain the light intensity value measured by the ambient light sensor and the humidity value measured by the humidity sensor as environmental features.

4. A method for judging display faults of an LCD display screen according to claim 1, characterized in that: The construction method of the multi-dimensional feature fault library described in Step 3 is as follows: Integrate the data collected and feature-extracted from each sensor to form a data set containing multiple feature vectors, and label the collected data. The labeling content includes whether there is a fault and the type of the fault. And select a database management system to store the multi-dimensional fault feature library, and design the database table structure, including a data table, a fault type table, and a sensor information table. The data table is used to store the collected data and the corresponding feature vectors. The fault type table records the information of different fault types. The sensor information table stores the relevant parameters and status information of the sensors.

5. A method for judging display faults of an LCD display screen according to claim 1, characterized in that: The multi-source signal synchronous acquisition and preprocessing described in Step 1 includes a timing correlation module. The timing correlation module includes a hardware clock synchronization unit, a software timestamp calibration unit, and an association rule mining unit. Hardware clock synchronization unit: In the multi-source data acquisition module, use a hardware clock synchronization device to synchronize the device clocks of each sensor with the standard time server through the NTP protocol; and set a timer interrupt program in the multi-source data acquisition module. Whenever the timer is triggered, send a synchronization signal to each sensor at the same time, so that each sensor records data and attaches a timestamp at the same moment. Software timestamp calibration unit: In the multi-source data acquisition module, for each collected data sample, record its acquisition timestamp, and calibrate the data acquisition timestamps of different sensors to make their frequencies the same. Association rule mining unit: Use the Apriori algorithm to discover the association rules between sensor data.

6. A method for judging display faults of an LCD display screen according to claim 5, characterized in that: The algorithm implementation steps of the Apriori algorithm are as follows: S1: Data acquisition and integration: Collect data from optical sensors, electrical signal probes, thermal imaging modules, and environmental sensors, and preprocess the collected data, including data cleaning, standardization, and discretization. S2: Transaction database construction: Regard each set of preprocessed data as a transaction. S3: Frequent item set mining: Include frequent item set mining of single-sensor data features and frequent item set mining of combined features of multiple-sensor data. S4: Association rule generation: For each frequent item set, generate all possible non-empty subset combinations as the antecedent and consequent of the rule, generate rules, and calculate the confidence of each rule; if the confidence meets the given minimum confidence threshold, retain the rule. S5: Rule optimization: Remove redundant rules.

7. A method for judging display faults of an LCD display screen according to claim 1, characterized in that: The dynamic test mode generation described in Step 4 includes a preliminary detection module, a dynamic test image generation module, and a reinforcement learning optimization module. Preliminary detection module: uses an optical sensor to collect the initial image of the display screen, pre-processes the image, and then uses a feature extraction algorithm to preliminarily determine the area where the fault may exist; Dynamic test image generation module: Based on the preliminary detection results, a targeted test image is generated through a deep neural network. This module contains multiple sub-networks for generating different types of test images, including a high-frequency flicker image generation sub-network, a gradient grayscale image generation sub-network, and a polar coordinate grid image generation sub-network. Each sub-network receives the output results of the preliminary detection module, including the possible location and type information of the fault area, and then generates a corresponding test image; Reinforcement learning optimization module: The Q-learning algorithm is used to optimize the generated test image sequence. This module records the execution effect of each test image and adjusts the order and selection of the test sequence based on these records to shorten the detection time while ensuring coverage.

8. A method for judging display faults of an LCD display screen according to claim 7, characterized in that: The training process of the high-frequency flicker image generation subnetwork is as follows: prepare a number of display screen image data with annotations, the annotation content includes the location and type of the fault area, and use 70% of the data as a training set and 30% of the data as a verification set; for each training sample, input the annotated fault area information into the neural network, calculate the difference between the output high-frequency flicker image and the actual annotated flicker image, and adjust the weight and bias of the neural network through the back propagation algorithm to minimize the difference; The training process of the gradient grayscale image generation subnetwork is as follows: collecting a number of display screen images of different fault types as training data, extracting fault area information from each image and annotating the corresponding gradient grayscale image, using a supervised learning method to train the neural network so that the output gradient grayscale image is as close as possible to the annotated image, and using a cross entropy loss function to measure the difference and adjust the model parameters during the training process; The training process of the polar coordinate grid image generation subnetwork is as follows: construct a training data set, which includes fault area information and corresponding polar coordinate grid image annotations, serialize the input data and input it into the RNN for training. During the training process, the model performance is evaluated by calculating the accuracy between the predicted polar coordinate grid image and the annotated image, and the model parameters are adjusted according to the loss value.

9. A method for judging display faults of an LCD display screen according to claim 7, characterized in that: The Q-learning algorithm optimizes the generated test image sequence in the following way: S1: State definition: The state space S consists of the following elements: the state information of the display area to be detected, including the results of the detected area and the characteristics of the remaining undetected area; the history of the selected test image sequence, including the type of test image, the execution order and the detection result of each test image in this state; S2: Action definition: The action space A includes selecting the next test image to be executed. After each action is selected, the corresponding test image is executed and its effect is observed; S3: Reward function design: The reward function R is determined based on the change in state after the action is executed. If the test image finds a new fault area, a positive reward is given. If the test image misjudges the fault area, a negative reward is given. S4: Q-table Update: Initialize the Q-table, set the Q-values of all state-action pairs to 0 or a small random value, and update the Q-table according to the actual reward after executing the action and the estimated future reward. S5: Policy Improvement: Select the optimal policy according to the updated Q-table. In each state, select the action with the highest Q-value as the optimal action.

10. An LCD display fault judgment system, characterized in that: It includes a multi-source data acquisition module, a central processor, a memory, and a user interface. The multi-source data acquisition module includes an optical sensor, an electrical signal probe, a thermal imaging module, and an environmental sensor and their data acquisition host. The optical sensor is a camera, which is used to collect the image data of the LCD display screen to obtain the display content, brightness, and color information of the screen. The electrical signal probe is a voltage and current probe, which monitors the driving voltage and current of the LCD display screen. The thermal imaging module is a thermal imager, which detects the temperature distribution of each area of the LCD display screen through thermal imaging technology. The environmental sensor is a light sensor and a humidity sensor, which collect environmental light intensity and humidity environmental data. Central Processor: Responsible for coordinating the work of each sensor, receiving the data transmitted by the sensors, and performing preliminary processing and analysis. Run the fault judgment algorithm, compare and judge the data according to the preset rules and models to determine whether there is a fault in the LCD display screen and the type and severity of the fault. Memory: Used to store system programs, fault feature libraries, and sensor calibration parameter data. The system program is a software instruction set that controls the operation of the entire system. The fault feature library contains the feature information of various common faults and corresponding handling measures. The sensor calibration parameters are used to ensure the accuracy and reliability of sensor measurements. User Interface: Used to set system parameters, start and stop data acquisition and fault judgment processes, view fault judgment results and historical records, realize the automatic control and operation management of the system, and automatically control the working status and data acquisition process of each hardware component according to the preset conditions and rules.

Images