LED display defect prediction and process adjustment method and system based on multi-modal fusion

Abstract

The invention relates to the technical field of LED display, solves the problem that the existing LED display defect detection and parameter adjustment technology is lack of multi-modal information fusion and intelligent process control capability and is difficult to meet the quality control requirement of a high-precision display product, and provides an LED display defect prediction and process adjustment method and system based on multi-modal fusion. The method comprises the following steps: performing multi-modal data fusion processing on optical image data, electrical test data and thermal infrared imaging data corresponding to a to-be-tested LED display screen to obtain fused data; inputting the fused data into a pre-trained defect recognition model to obtain a defect recognition result; according to a process parameter adjustment strategy corresponding to the defect identification result, adjusting the original process parameter to obtain a target process parameter; and according to the target process parameters, process flow correction processing is carried out, and a qualified LED display screen is produced. According to the method, the defect identification precision is improved, and the quality control requirement of high-precision LED display screen production is met.

Description

Technical Field

[0001] The present invention relates to the field of LED displays, and in particular to a method and system for LED display defect prediction and process adjustment based on multimodal fusion. Background Art

[0002] LED displays are a type of flat-panel display technology based on light-emitting diode arrays. They offer advantages such as high brightness, fast response, low energy consumption, and a long service life. They are widely used in a variety of fields, including outdoor advertising, stage performances, traffic information dissemination, and smart manufacturing. With the development of high-density, high-resolution packaging technologies such as Mini-LED and Micro-LED, LED displays have placed higher demands on display performance, including brightness uniformity, color reproduction, and response consistency. However, in actual production, LED displays are prone to a variety of complex defects, such as uneven brightness, color cast, dead lights, poor solder joints, and flicker, due to factors such as chip processing, package offset, driver circuit anomalies, or uneven heat dissipation. Therefore, high-precision, full-coverage defect detection for LED displays has become a critical step in ensuring display quality and production yield.

[0003] Currently, the industry primarily relies on visible light image processing or manual visual inspection to detect defects in LED displays. Common techniques include using brightness distribution maps acquired by CCD or CMOS image sensors to identify visual defects using traditional image processing algorithms such as image enhancement, edge detection, and texture analysis, or classification models based on convolutional neural networks (CNNs). Furthermore, some systems also incorporate spectral analysis, grayscale testing, or spot-measurement equipment to collect local grayscale brightness data for analyzing brightness non-uniformity. Electrical performance testing typically uses an oscilloscope or test bench to monitor the voltage and current waveforms output by the driver circuit. Thermal testing typically relies on infrared thermal imagers to monitor the entire display's thermal distribution to identify thermal runaway or soldering anomalies. While these detection methods offer some recognition capabilities in their respective dimensions, existing technologies generally suffer from the following technical limitations: a lack of joint modeling of optical, electrical, and thermal information makes it difficult to identify and accurately locate the root causes of complex defects such as MURA-type brightness unevenness, thermal drive anomalies, and intermittent faults caused by poor soldering. For example, dim brightness in a particular area could be caused by current fluctuations in the driver IC or insufficient brightness in the packaged chip itself. Images alone cannot determine the true cause, making it impossible to accurately match the corresponding process parameters for correction. Furthermore, existing solutions generally lack an automated closed-loop process adjustment mechanism, failing to integrate identified defect types with historical process parameter records for intelligent feedback control. This results in a repair process that relies heavily on manual experience, resulting in low efficiency and poor consistency.

[0004] Therefore, how to achieve accurate identification of the causes of multi-source defects in LED displays and intelligent linkage adjustment of corresponding process parameters is a technical problem that needs to be solved urgently. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides an LED display defect prediction and process adjustment method and system based on multimodal fusion, which is used to solve the problem that the existing LED display defect detection and parameter adjustment technology lacks multimodal information fusion and intelligent process control capabilities when facing high-complexity, multi-factor coupled defect identification tasks, and is difficult to meet the quality control requirements of high-precision display products.

[0006] An embodiment of the present invention provides a method for LED display defect prediction and process adjustment based on multimodal fusion, the method comprising:

[0007] Obtain optical image data, electrical test data, and thermal infrared imaging data corresponding to the LED display to be tested;

[0008] Performing multimodal data fusion processing on the optical image data, electrical test data, and thermal infrared imaging data to obtain fused data;

[0009] Inputting the fused data into a pre-trained defect recognition model to obtain a defect recognition result;

[0010] According to the process parameter adjustment strategy corresponding to the defect identification result, the original process parameters corresponding to the LED display screen to be tested are adjusted to obtain the target process parameters;

[0011] According to the target process parameters, the process flow of the LED display screen to be tested is corrected to produce a qualified LED display screen that meets the quality requirements.

[0012] Preferably, the step of obtaining optical image data, electrical test data, and thermal infrared image data corresponding to the LED display screen to be tested includes:

[0013] Testing the LED display screen to be tested according to preset standard test conditions to obtain the real-time test status of the LED display screen to be tested;

[0014] When the real-time test state reaches a preset test state, initial optical image data, initial electrical test data, and initial thermal infrared imaging data corresponding to the LED display screen to be tested are obtained;

[0015] Performing time stamping and synchronization processing on the optical image initial data, the electrical test initial data, and the thermal infrared imaging initial data to obtain original multimodal data, wherein the original multimodal data includes original optical image data;

[0016] Performing lens distortion correction and geometric transformation processing on the original optical image data to obtain standard optical image data;

[0017] Performing affine transformation registration processing on the standard optical image data and the thermal infrared imaging initial data according to preset marking points to obtain spatially aligned multimodal image data;

[0018] Performing drive signal period alignment processing on the multimodal image data and the electrical test initial data to obtain multimodal test data;

[0019] The multimodal test data is normalized to obtain the optical image data, electrical test data and thermal infrared imaging data.

[0020] Preferably, the performing multimodal data fusion processing on the optical image data, electrical test data and thermal infrared imaging data to obtain fused data includes:

[0021] Performing feature extraction on the optical image data, electrical test data, and thermal infrared imaging data to obtain optical feature vectors, electrical feature vectors, and thermal feature vectors;

[0022] Performing cross-modal correlation analysis on the optical eigenvector, the electrical eigenvector, and the thermal eigenvector to obtain a correlation weight matrix between the modes;

[0023] performing feature fusion processing on the optical feature vector, the electrical feature vector, and the thermal feature vector according to the association weight matrix to obtain a primary fusion feature;

[0024] Performing spatiotemporal consistency verification on the primary fusion features to obtain optimized multimodal fusion features;

[0025] Performing dimension reduction and standardization processing on the multimodal fusion features to obtain the fusion data.

[0026] Preferably, the extracting features of the optical image data, the electrical test data and the thermal infrared imaging data to obtain the optical feature vector, the electrical feature vector and the thermal feature vector respectively comprises:

[0027] Performing convolution feature extraction processing on the optical image data to obtain the optical feature vector including brightness distribution, chromaticity deviation and defect outline;

[0028] Performing short-time Fourier transform processing on the electrical test data to obtain the electrical feature vector including harmonic components, current fluctuation characteristics, and voltage ripple characteristics;

[0029] The thermal infrared imaging data is subjected to gradient field analysis processing to obtain the thermal characteristic vector including temperature gradient, hot spot distribution and heat conduction characteristics.

[0030] Preferably, inputting the fused data into a pre-trained defect recognition model to obtain a defect recognition result comprises:

[0031] Performing channel-by-channel feature extraction processing on the fused data to obtain a multimodal initial feature set;

[0032] Based on the multimodal initial feature combination, inter-modal attention modeling and alignment processing are performed on each feature channel to obtain a fused feature tensor;

[0033] According to the preset light color deviation template and defect instance database, the fused feature tensor is matched to obtain a target feature vector corresponding to the preset defect label;

[0034] According to the target feature vector, the fused feature tensor is input into the defect recognition model for forward recognition processing to obtain the defect recognition result, wherein the defect recognition result includes the defect type, defect location and defect level.

[0035] Preferably, the process parameter adjustment strategy corresponding to the defect identification result is used to adjust the original process parameters corresponding to the LED display screen to be tested to obtain the target process parameters, including:

[0036] According to the defect type, a preset historical process parameter adjustment strategy library is searched and associated to obtain a parameter adjustment strategy set corresponding to the defect type;

[0037] According to the defect location, performing spatial positioning mapping processing on the parameter adjustment strategy set to obtain a local process adjustment area corresponding to the defect location;

[0038] According to the defect level, performing amplitude correction processing on the adjustment strategy of the local process adjustment area to obtain an adjustment intensity parameter that matches the defect level;

[0039] The original process parameters are adjusted according to the adjustment intensity parameters and the local process adjustment area to obtain the target process parameters.

[0040] Preferably, the process of performing an associated search on a preset historical process parameter adjustment strategy library according to the defect type to obtain a parameter adjustment strategy set corresponding to the defect type includes:

[0041] Performing semantic coding and standard classification processing on the defect type to obtain defect label information;

[0042] According to the defect label information, indexing is performed on a preset defect-process strategy mapping index table to obtain a candidate set of historical process parameter adjustment records;

[0043] Based on the candidate set of historical process parameter adjustment records, cluster analysis is performed on the process parameter change dimensions in the records to obtain a set of parameter adjustment paths that are highly correlated with the defect type;

[0044] The parameter adjustment path set is weighted and sorted to obtain a priority-sorted defect process parameter adjustment strategy set.

[0045] Preferably, performing spatial positioning mapping processing on the parameter adjustment strategy set according to the defect location to obtain a local process adjustment area corresponding to the defect location includes:

[0046] According to the defect location, each pixel point in the display screen to be tested is partitioned to obtain the target pixel area where the defect is located;

[0047] According to the position tag corresponding to the target pixel area, obtaining a hardware control unit identifier corresponding to the position tag;

[0048] Filtering the parameter adjustment strategy set according to the hardware control unit identifier to obtain the local adjustment strategy subset;

[0049] According to the effective area parameters in the local adjustment strategy subset, a display screen control interface or a driver chip control instruction is subjected to spatial offset mapping processing to obtain a local process adjustment parameter;

[0050] According to the local process adjustment parameters and in combination with the defect level information, an adjustable expansion process is performed on the boundary of the process adjustment region to obtain the local process adjustment region.

[0051] Preferably, performing amplitude correction processing on the adjustment strategy of the local process adjustment area according to the defect level to obtain an adjustment intensity parameter matching the defect level includes:

[0052] Performing a level quantization mapping process on the defect level to obtain a weight coefficient corresponding to the defect level;

[0053] Normalizing each adjustment parameter according to the initial adjustment strategy parameters in the local process adjustment area to obtain a reference value of the adjustable range;

[0054] Modifying the initial adjustment strategy parameters according to the weight coefficient and the reference value to obtain initial adjustment intensity parameters that match the defect level;

[0055] According to the maximum allowable adjustment range corresponding to the initial adjustment strength parameter, an amplitude threshold limiting process is performed on it to obtain the adjustment strength parameter that meets the safety control boundary.

[0056] In second aspect, an embodiment of the present invention provides an LED display defect prediction and process adjustment system based on multimodal fusion, the system comprising at least one processor, at least one memory, and computer program instructions stored in the memory, and when the computer program instructions are executed by the processor, the method described in the first aspect is implemented.

[0057] In summary, the beneficial effects of the present invention are as follows:

[0058] The embodiment of the present invention provides a method and system for LED display defect prediction and process adjustment based on multimodal fusion, the method comprising: obtaining optical image data, electrical test data, and thermal infrared imaging data corresponding to the LED display screen to be tested; performing multimodal data fusion processing on the optical image data, electrical test data, and thermal infrared imaging data to obtain fused data; inputting the fused data into a pre-trained defect recognition model to obtain a defect recognition result; adjusting the original process parameters corresponding to the LED display screen to be tested according to a process parameter adjustment strategy corresponding to the defect recognition result to obtain target process parameters; and performing process flow correction processing on the LED display screen to be tested according to the target process parameters to produce a qualified LED display screen that meets quality requirements. The present invention introduces three modal data types: optical image, electrical test, and thermal infrared imaging, performs multi-dimensional detection on the LED display screen, utilizes technologies such as time synchronization and spatial registration to achieve data alignment, and generates fused feature data through processing methods such as feature extraction, cross-modal fusion, and attention modeling to enable the defect recognition model to accurately analyze the defect type, location, and level. Compared with the traditional technical path that only relies on a single modality or manual rule judgment, the present invention can effectively deal with complex defects caused by the coupling of multiple factors such as driver IC, current fluctuation, pixel failure, etc., and further link the historical process strategy library, spatial positioning mapping mechanism and level control mechanism according to the recognition results to intelligently generate target process parameters, realize dynamic correction and closed-loop optimization of local or global process flows, significantly improve the accuracy and robustness of defect recognition, and at the same time give the system a high degree of automatic adjustment and adaptive control capabilities, breaking through the key bottlenecks of existing LED defect detection and process adjustment methods when facing high-complexity display tasks, such as insufficient multi-modal fusion and delayed adjustment response, and meeting the higher requirements for quality control in the production of high-precision LED displays. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work, and these are all within the scope of protection of the present invention.

[0060] Figure 1 1 is a schematic diagram of the overall process of the LED display defect prediction and process adjustment method based on multimodal fusion in an embodiment of the present invention;

[0061] Figure 2 is a schematic diagram of a process for performing multimodal data fusion processing on the optical image data, electrical test data, and thermal infrared imaging data to obtain fused data in an embodiment of the present invention;

[0062] Figure 3 It is a flow chart of inputting the fused data into a pre-trained defect recognition model to obtain a defect recognition result in an embodiment of the present invention.

[0063] Figure 4 It is a structural diagram of an LED display defect prediction and process adjustment system based on multimodal data fusion in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the objects, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present invention by illustrating examples of the present invention.

[0065] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0066] It should be noted that all actions of acquiring signals, information or data in the present invention are performed in compliance with the corresponding local data protection laws and policies and with authorization from the corresponding device owner.

[0067] Example 1

[0068] See Figure 1 The embodiment of the present invention provides a method for LED display defect prediction and process adjustment based on multimodal fusion, the method comprising:

[0069] Obtain optical image data, electrical test data, and thermal infrared imaging data corresponding to the LED display to be tested;

[0070] Specifically, in the quality inspection process of Mini-LED displays, obtaining multi-modal data is the basis for achieving accurate defect identification. Among them, optical image data refers to visual parameters such as display image brightness, chromaticity, and uniformity collected using a calibration camera; electrical test data includes data related to circuit functions such as voltage, current, and driver IC status; thermal infrared imaging data is a temperature distribution image of the LED panel or driver module under working conditions collected using a thermal imager. By jointly collecting these three types of data, a full-dimensional detection foundation can be constructed that includes visual performance, circuit status, and thermal characteristics. The purpose is to provide a comprehensive and objective data foundation for subsequent defect identification and classification. Single-modal data (such as relying solely on images) often cannot accurately locate the cause of complex defects. For example, image defects that also manifest as uneven brightness may be caused by defects in the LED chip itself, unstable current, or poor heat dissipation. By obtaining the above multi-modal data, root cause tracing and defect source tracing can be achieved in subsequent analysis, thereby improving recognition accuracy and judgment reliability, and supporting a more intelligent quality judgment system.

[0071] Preferably, the step of obtaining optical image data, electrical test data, and thermal infrared image data corresponding to the LED display screen to be tested includes:

[0072] Testing the LED display screen to be tested according to preset standard test conditions to obtain the real-time test status of the LED display screen to be tested;

[0073] Specifically, standard test conditions refer to a set of unified test specifications set to ensure the consistency, repeatability and scientific nature of various display performance index tests in LED display quality inspection. The standard usually includes test environment temperature (such as 25°C), humidity, power supply voltage range, display content (such as R / G / B / W full-screen test pattern), display brightness level, refresh rate, viewing distance, etc.; for example, to test color consistency and brightness uniformity, it may be required to light up a white screen with 100% brightness on the display in a darkroom and use a scientific-grade calibrated camera to shoot it, in order to ensure the stability and comparability of data acquisition. Only when the LED display is in a standardized operating state can the validity and representativeness of optical, electrical, thermal and other test data be guaranteed, and data deviations caused by external environment or random operating state can be avoided, thereby ensuring the reliability of defect identification results. Real-time test status monitoring is precisely to determine whether the LED screen is in a standard working state, such as ensuring that the image output is stable, the current waveform is smooth, and the chip is in full brightness. During the implementation process, the LED display is first powered on by an automatic control device, which drives it to display a standard pattern, such as red, green, blue, and white, frame by frame, according to a preset process. Simultaneously, a sensor system begins collecting key operating indicators in real time, such as current stability, voltage fluctuation range, temperature rise curve, and screen refresh completion. Backend logic compares this real-time data with standard test conditions, determining, for example, whether the current fluctuates smoothly within a ±2% range, whether the screen brightness consistently reaches the target value, and whether the temperature rise exceeds a threshold. By introducing a real-time status monitoring mechanism, data errors caused by unstable display status before data collection are collected are effectively avoided, laying a solid foundation for subsequent data quality. Furthermore, this real-time monitoring and standard comparison mechanism can quickly identify LED displays with abnormal startup or malfunctioning, allowing problematic displays to be excluded or recorded before formal testing begins, significantly improving overall testing efficiency and accuracy.

[0074] When the real-time test state reaches a preset test state, initial optical image data, initial electrical test data, and initial thermal infrared imaging data corresponding to the LED display screen to be tested are obtained;

[0075] Specifically, the real-time test state reaching the preset test state means that the current operating state of the LED screen is detected to meet the standard settings. For example, the screen is displayed stably for more than 3 seconds, the brightness meets the standard, the voltage and current fluctuations are normal, and there are no short-circuit faults in the chip. At this time, the test can be considered to have entered the sampling state; the initial data refers to the first round of raw data collected under the standard state, which is used as a benchmark for subsequent processing, comparison or model input; the purpose is to accurately collect multimodal data to ensure that the collected data truly reflects the normal performance of the LED screen and avoid distorted information collected due to unstable state or noise interference. In the specific implementation process, after determining that the screen operating state meets the preset standards, multiple sensors are automatically started synchronously: the CIE-XYZ correction camera collects the optical image of the current display screen, including RGB brightness, color temperature, brightness uniformity, etc.; the voltage and current monitoring equipment records the voltage, current waveform, and power value of the current driving circuit; the thermal infrared camera obtains infrared thermal images of the panel and backplane chip area, including hot spot distribution, temperature gradient, etc.; the three data are recorded separately and labeled with the same timestamp for subsequent fusion and registration processing. This step boasts a strong system collaborative sampling capability. Through a multi-sensor collaborative triggering mechanism, it ensures that all types of data are captured within the optimal time window while avoiding the timing errors introduced by manual sampling. This significantly improves the consistency and alignment of multimodal data, particularly during the data fusion phase, significantly reducing errors and improving the generalization and recognition accuracy of the trained model, thereby enhancing the industrial applicability of the overall defect recognition system.

[0076] The optical image initial data, the electrical test initial data and the thermal infrared imaging initial data are subjected to time synchronization and spatial registration processing to obtain the optical image data, the electrical test data and the thermal infrared imaging data.

[0077] Specifically, by performing time synchronization and spatial registration on the initial optical image data, the initial electrical test data, and the initial thermal infrared imaging data, unified alignment of multimodal data in both the temporal and spatial dimensions is achieved. Time synchronization ensures that all types of data correspond to the same display state through unified timestamp marking and drive cycle alignment. Spatial registration aligns image, thermal map, and electrical signal data in the same physical coordinate system through lens distortion correction, geometric transformation, and affine alignment of marker points. This processing provides a precise and consistent data foundation for subsequent multimodal fusion analysis, helping to accurately identify defect locations and causes, and improving the system's detection reliability and intelligence level.

[0078] Preferably, the performing time synchronization and spatial registration processing on the initial optical image data, the initial electrical test data, and the initial thermal infrared imaging data to obtain the optical image data, the electrical test data, and the thermal infrared imaging data includes:

[0079] Performing time stamping and synchronization processing on the optical image initial data, the electrical test initial data, and the thermal infrared imaging initial data to obtain original multimodal data, wherein the original multimodal data includes original optical image data;

[0080] Specifically, different types of data (such as optical images, electrical waveforms, and thermal images) often originate from different sensors, each with its own independent sampling frequency and clock mechanism. Therefore, all types of initial data must be timestamped and synchronized. Timestamping involves adding a time tag in a unified format to each frame of image or each set of electrical data, identifying the moment of acquisition and facilitating subsequent alignment. The goal of synchronization is to unify the data collected by multiple sensors in the temporal dimension, thereby constructing raw multimodal data—that is, a collection of three types of data at the same time: images, telecommunications, and thermal images. This ensures strict temporal consistency between the multimodal data, enabling comparability and consistency in subsequent spatial registration and model fusion analysis. For example, if the optical image is acquired when the LED screen is on and the current waveform is acquired when it is off, the causal relationship between the two data will be lost, leading to misjudgment. During implementation, a high-precision clock synchronization module (such as a GPS clock or industrial synchronized clock module) is configured to synchronize the data acquisition systems of all sensors. Each data packet is automatically timestamped with a millisecond-level timestamp. Subsequently, by comparing timestamps and performing interpolation or resampling operations, the continuous waveform of the electrical data is aligned with the discrete frame image in the temporal dimension, extracting multiple slices of optical, electrical, and thermal data synchronized within the same time window. This step has strong temporal coordination characteristics and its technical advantage lies in eliminating the modal misalignment caused by sensor sampling differences, significantly improving the temporal accuracy of multimodal analysis.

[0081] Performing lens distortion correction and geometric transformation processing on the original optical image data to obtain standard optical image data;

[0082] Specifically, due to the spherical structure or wide-angle distortion of the camera lens, the raw optical image captured may exhibit curved edges and distorted proportions, a phenomenon known as lens distortion. Furthermore, the relative angle and distance between the test device and the display screen may also introduce viewing angle deviations. Therefore, lens distortion correction and geometric transformation are required to restore the raw optical image to a standard form consistent with the actual display structure. Ultimately, standard optical image data is obtained—data that accurately reflects the physical pixel arrangement in terms of spatial scale and shape after processing. First, image dedistortion is performed using known camera calibration parameters (such as the intrinsic parameter matrix and distortion coefficients). Common algorithms include the undistort function in OpenCV. Then, based on the geometric relationship between the actual shooting angle and the target display area, perspective correction, affine transformation, or projection transformation is performed to align each pixel in the image as closely as possible with the actual pixel. This process can improve accuracy by employing methods such as calibration plates and laser-assisted marking to ensure good spatial geometric consistency of the optical image, thereby avoiding defect recognition errors caused by image distortion. This step enhances the structural credibility of the image data and lays an accurate spatial foundation for subsequent registration and model fusion.

[0083] Performing affine transformation registration processing on the standard optical image data and the thermal infrared imaging initial data according to preset marking points to obtain spatially aligned multimodal image data;

[0084] Specifically, because thermal infrared images and standard optical images originate from different sensors and may be mounted at different angles, distances, or resolutions, their spatial resolution often suffers from scale inconsistencies, perspective offsets, and displacement misalignments. To achieve spatial alignment between these two types of images, affine transformation registration is performed based on preset markers (such as calibration targets, border positioning points, and luminous pixels). Ultimately, the two images are spatially aligned at the pixel level, thereby constructing spatially aligned multimodal image data. The goal is to establish a one-to-one correspondence between the thermal image data and the optical image, thereby achieving corresponding representation of the same defect in both modalities. Fixed markers are designed within the display area, such as the four screen corners, a crosshair target, and edge highlights. The pixel coordinates of these markers are identified in both the thermal infrared and optical images. The thermal infrared image is then affinely transformed to the standard optical image coordinate system by calculating the affine matrix between the two sets of coordinates (including translation, rotation, and scaling transformations). Optimization techniques such as refined multiscale registration and feature point RANSAC filtering can be used to improve registration accuracy if necessary.

[0085] Performing drive signal period alignment processing on the multimodal image data and the electrical test initial data to obtain multimodal test data;

[0086] Specifically, the display content of LED displays changes periodically, controlled by the driver chip, which repeatedly loads pixel data at a specific refresh rate (e.g., every 16.6ms). Electrical test data (such as current and voltage waveforms) is essentially a continuous time series signal. Therefore, aligning it with the periodic image data requires drive signal cycle alignment. This involves dividing the electrical data into drive cycle segments based on the screen refresh rhythm and correlating them with corresponding image frames to generate complete multimodal test data. The goal is to align the time-content mapping of optical and electrical information. For example, ensuring that a peak in the current waveform corresponds to a high-brightness frame rather than a transitional frame, thus avoiding misjudgments caused by timing discrepancies. This is achieved by embedding synchronization signals—such as PWM clock edges, frame start signals, or external trigger pulses—in the LED drive process as the boundaries for electrical waveform segmentation. The system then extracts image and thermal map frames within each cycle and associates them with the corresponding electrical signal segments for unified binding. Signal envelope alignment and frequency domain phase comparison can also be performed to optimize timing accuracy if necessary.

[0087] The multimodal test data is normalized to obtain the optical image data, electrical test data and thermal infrared imaging data.

[0088] Specifically, before fusing multimodal data from different sources, they need to be normalized. This means mapping the value range and distribution of each data channel to a unified scale so that the machine learning model can process feature information at the same scale. Data from different modalities vary greatly. For example, an image pixel value may be 0255, a current waveform may be 03A, and a temperature value may be 20-80°C. If not processed uniformly, the model may be biased towards one modality, affecting the accuracy of the discrimination. The purpose is to improve the model's fusion capability and training stability. Through normalization, data from different modalities are pulled back into a unified feature space to achieve equal-weighted input or weighted adaptive input. Normalization not only speeds up convergence but also reduces the problem of gradient explosion or vanishing caused by differences in data scale. In specific implementations, optical images can be linearly normalized (e.g., divided by 255), temperature data can be linearly scaled to the [0, 1] interval according to an empirical interval (e.g., 20-100°C), and electrical waveforms can be normalized using maximum value or Z-score (mean is 0, standard deviation is 1). In addition, nonlinear normalization methods such as logarithmic transformation and piecewise normalization can also be introduced to improve adaptability.

[0089] Performing multimodal data fusion processing on the optical image data, electrical test data, and thermal infrared imaging data to obtain fused data;

[0090] Specifically, by performing multimodal data fusion processing on optical image data, electrical test data, and thermal infrared imaging data, the three types of feature information with complementary characteristics are integrated into unified fusion data to form a more comprehensive expression of the status of the LED display screen. Fusion methods may include feature-level fusion (such as splicing optical feature vectors, electrical feature vectors, and thermal feature vectors or weighted combination through an attention mechanism) or decision-level fusion (such as joint voting after independent judgment of each modality). The purpose of this step is to combine the advantages of each modality in spatial distribution, electrical properties, and thermal response, enhance the ability to identify and locate complex defects, effectively improve the stability and accuracy of the system under multi-factor interference, and provide a high-quality, context-related input basis for subsequent defect classification and quality judgment.

[0091] Preferably, see Figure 2 , the multimodal data fusion processing is performed on the optical image data, electrical test data and thermal infrared imaging data to obtain fused data including:

[0092] Performing feature extraction on the optical image data, electrical test data, and thermal infrared imaging data to obtain optical feature vectors, electrical feature vectors, and thermal feature vectors;

[0093] Specifically, the original optical image data, electrical test data, and thermal infrared imaging data are converted into discriminative structured feature vectors for subsequent use in defect identification and classification. Specifically, the optical image data is extracted through methods such as convolutional neural networks to extract visual features such as brightness distribution, chromaticity deviation, and edge contours to form optical feature vectors; the electrical test data is extracted through time-frequency analysis methods such as short-time Fourier transform to generate electrical feature vectors; the thermal infrared imaging data is extracted through gradient field analysis to extract temperature gradients, hot spot structures, and thermal diffusion characteristics to construct thermal feature vectors. By extracting the core characterization features of each modality separately, this step lays a solid feature foundation for multimodal data fusion and intelligent defect judgment, and improves the accuracy and robustness of the overall recognition system.

[0094] Preferably, the extracting features of the optical image data, the electrical test data and the thermal infrared imaging data to obtain the optical feature vector, the electrical feature vector and the thermal feature vector respectively comprises:

[0095] Performing convolution feature extraction processing on the optical image data to obtain the optical feature vector including brightness distribution, chromaticity deviation and defect outline;

[0096] Specifically, convolutional feature extraction involves performing local perception operations on an image using a convolutional neural network (CNN) or traditional filters to extract multi-level visual features such as texture, edges, and color variations. Visible light images record the brightness and color representation of the screen display. Therefore, convolution operations can be used to extract brightness distribution (such as the uniformity of bright and dark areas), chromaticity deviation (such as deviation from the standard D65 color temperature), and defect outlines (such as edge structures such as bright spots, dark spots, and moiré). The two-dimensional image data is converted into discriminative optical feature vectors, enabling subsequent classification models to perceive the image representation of specific defect patterns. Using only the raw image as input is susceptible to interference from lighting conditions and shooting angles, while features extracted through convolution are translationally invariant and locally perceptive, enabling more stable descriptions of defects such as brightness anomalies and edge cracks. During implementation, pre-trained deep convolutional networks (such as ResNet and MobileNet) or custom shallow networks are used to perform multi-layer convolution processing on the optical image data. The lower layers capture basic textures such as edges and corners, the middle layers extract brightness distribution and color trends, and the upper layers focus on regional structural features. Finally, a pooling operation is used to compress the feature map and flatten it into a vector, namely the "optical feature vector", which effectively extracts key image performance indicators while filtering out background and irrelevant information, improving the model's recognition accuracy and robustness for image defects.

[0097] Performing short-time Fourier transform processing on the electrical test data to obtain the electrical feature vector including harmonic components, current fluctuation characteristics, and voltage ripple characteristics;

[0098] Specifically, in electrical signal analysis, the short-time Fourier transform (STFT) is a method for converting nonstationary signals into a time-frequency domain representation. It performs a local Fourier transform by sliding a window over the signal, revealing how the signal's frequency varies over time. In LED driver circuit testing, the STFT can reveal current fluctuation characteristics and harmonic components (high-frequency components and distortion caused by poor rectification) in current signals, as well as voltage ripple (periodic, micro-amplitude jitter caused by unstable power supply or driver chip mismatch) in voltage signals. Converting the original voltage and current time series waveforms into a feature vector that represents both the frequency and time domains allows the system to identify periodic anomalies, jumps, or noise interference that cannot be directly identified from the raw signals. For example, the same current waveform may appear stationary in the time domain but exhibit high-frequency perturbations in the frequency domain, indicating anomalies in certain driver ICs or filter components. In implementation, a window function (such as a Hamming window) is first applied to the electrical sampling data (e.g., thousands of points collected within a second). A fast Fourier transform (FFT) is then performed on each window, ultimately resulting in a set of time-frequency spectrograms. Statistical indicators such as the distribution of major harmonic energy, current fluctuation amplitude, voltage ripple amplitude, and frequency content are extracted and converted into fixed-length electrical feature vectors. This reveals signal stability and quality through a frequency domain perspective. Its technical features include high-sensitivity, high-resolution electrical signal pattern mining capabilities, which help detect subtle drive anomalies that are difficult to detect with traditional mean analysis. This provides data for diagnosing issues such as cold solder joints, PCB routing anomalies, and filter failures, thereby enhancing the depth and accuracy of system fault diagnosis.

[0099] The thermal infrared imaging data is subjected to gradient field analysis processing to obtain the thermal characteristic vector including temperature gradient, hot spot distribution and heat conduction characteristics.

[0100] Specifically, thermal infrared imaging records the surface temperature distribution image of the LED panel under working conditions, while gradient field analysis mathematically models and quantifies the temperature change trend in the thermal image. Specifically, the gradient reflects the speed and direction of temperature change in space, which is suitable for describing phenomena such as local heating and heat diffusion direction. On this basis, key temperature gradient characteristics (local temperature rise intensity), hot spot distribution characteristics (high temperature point concentration and distribution range), and heat conduction characteristics (symmetry and diffusion shape of conduction from the central heat source to the surrounding area, etc.) can be extracted. Thermophysical changes that are highly correlated with defects in the thermal image are quantitatively extracted and converted into thermal feature vectors that can be used for model recognition. For example, problems such as LED chip cold soldering, driver IC overload, and poor contact of the heat sink will all appear as non-uniform or abnormally high temperature areas. These anomalies are difficult to determine using average temperature or maximum temperature alone, and their morphology must be determined based on gradient and structural information. During implementation, temperature normalization is first performed on the thermal image, and then the temperature gradient vector field of the image is calculated (such as using Sobel, Scharr and other operators to find the gradient). Structural features such as the gradient direction histogram, hotspot patch area distribution, and thermal center-boundary distance distribution can be further generated. The thermal map is encoded and compressed through principal component analysis (PCA) or CNN to form a fixed-length thermal feature vector that describes the thermal diffusion behavior. It has obvious structural thermal analysis characteristics and can identify micro-defects such as uneven heating and micro-thermal faults based on temperature change trends that cannot be identified by traditional infrared averaging methods. Combining visual features with electrical signal features, this thermal feature provides an important dimension for trimodal defect fusion judgment, providing support for the system to achieve accurate, full-stack intelligent judgment of LED quality.

[0101] Performing cross-modal correlation analysis on the optical eigenvector, the electrical eigenvector, and the thermal eigenvector to obtain a correlation weight matrix between the modes;

[0102] Specifically, cross-modal correlation analysis is performed on the optical, electrical, and thermal feature vectors. Cross-modal correlation analysis involves modeling the statistical or semantic relationships between feature vectors from different data modalities, aiming to reveal their inherent connections when representing the same physical defect. The resulting association weight matrix is ​​used to quantify the influence or contribution of each modal feature to the features of another modality. For example, whether an abnormal brightness region also appears as a hotspot in a thermal image, or whether a current anomaly corresponds to flicker in an optical image, is used to provide semantic and structural alignment for subsequent feature fusion. This avoids information redundancy or bias interference caused by directly concatenating heterogeneous features, thereby improving the quality and contextual consistency of the fused features. In implementation, pairwise interaction modeling of the optical, electrical, and thermal feature vectors is performed based on methods such as attention mechanisms, mutual information maximization, or canonical correlation analysis. For example, a pair of optical and thermal vectors is input, and a bidirectional attention mechanism is used to extract the matching strength matrix between them as part of the association weight matrix. Ultimately, the weights between the modalities are aggregated to form a complete inter-modal association weight matrix.

[0103] performing feature fusion processing on the optical feature vector, the electrical feature vector, and the thermal feature vector according to the association weight matrix to obtain a primary fusion feature;

[0104] Specifically, after obtaining the inter-modal correlation weight matrix, the optical feature vector, electrical feature vector and thermal feature vector will be weighted and combined according to the correlation weight matrix to achieve guided feature fusion processing and output a unified primary fusion feature. Fusion is no longer a simple splicing of different modal features, but rather an adjustment of the weight of each dimension through the correlation weight, so that the key dimensions of different modalities are more expressive, while redundant or unimportant dimensions are weakened or suppressed. The purpose is to generate a fusion representation with cross-modal collaborative perception capabilities, which can more completely reflect the multi-dimensional features of the target defect (visual performance, electrical anomalies and thermal distribution) and adapt to the subsequent classification model or regression model to judge the defect level. Implementation methods include attention-based weighted fusion, tensor fusion, or graph structure fusion. Taking the attention mechanism as an example, each dimension in the optical vector will be dynamically adjusted according to the attention score between it and the thermal or electrical vector, and finally form a fusion feature vector with comprehensive perception capabilities as the primary fusion feature.

[0105] Performing spatiotemporal consistency verification on the primary fusion features to obtain optimized multimodal fusion features;

[0106] Specifically, since multimodal data may come from different time windows and there are slight errors in spatial distribution, there is a risk of spatiotemporal inconsistency in the features after direct fusion, that is, the electrical features at a certain moment may be aligned with the image or heat map of an earlier frame, and a slight offset in space will also cause inconsistent focusing. Therefore, this step introduces a spatiotemporal consistency check process to correct the primary fusion features and obtain optimized multimodal fusion features. The implementation method includes temporal consistency check (such as timestamp-based sliding window matching), spatial consistency check (such as the matching deviation between the heat map and the center of the bright spot is less than a threshold), and the use of the spatiotemporal attention mechanism to re-weight the fused feature vector. At the same time, the spatial affine matrix can also be introduced to compensate for the position of the feature dimension or focus on the region to improve the spatial semantic consistency.

[0107] Performing dimension reduction and standardization processing on the multimodal fusion features to obtain the fusion data.

[0108] Specifically, the optimized multimodal fusion features are usually high-dimensional in structure and may contain redundant or weakly correlated information. Therefore, before being used as model input, dimensionality reduction and standardization are required. Dimensionality reduction (such as principal component analysis PCA, linear discriminant analysis LDA, autoencoder, etc.) can compress data dimensions and extract the most discriminative main features; while standardization (such as Z-score standardization, Min-Max normalization) is used to unify feature scales to avoid biasing the model due to a large numerical range of a certain dimension of features. The purpose of this step is to improve the learnability and model efficiency of the fused data, so that the subsequent defect recognition model can converge faster in a lower-dimensional space, while ensuring the consistency and balance of the input features at the numerical level, reducing the risk of training instability or overfitting. In specific implementation, the covariance matrix of the multimodal fusion features is first calculated, and its principal components are retained or the feature dimensions are compressed using a transformation network. Then, each feature dimension is normalized so that its mean is 0 and its standard deviation is 1, or stretched to the [0,1] interval. Finally, the fused data is output as the model input.

[0109] Inputting the fused data into a pre-trained defect recognition model to obtain a defect recognition result;

[0110] Specifically, the fused data is input into the pre-trained defect recognition model, and the corresponding defect recognition results are output to realize intelligent defect judgment of the Mini-LED display screen based on multimodal perception. The fused data is a highly integrated representation of optical, electrical and thermal features, and has the ability to comprehensively reflect complex information such as screen brightness, driving status and heat distribution. The pre-trained defect recognition model usually adopts a deep neural network (such as a multi-layer perceptron, convolutional network, or Transformer architecture), which has been trained on a large amount of labeled defect data and can accurately identify common problems such as uneven brightness, poor soldering, abnormal voltage ripple or chip overheating. The core purpose of this step is to use the learning ability of the model to realize the automatic recognition of complex and nonlinear defect features, thereby replacing manual visual inspection and improving detection efficiency and consistency. At the same time, the process has scalability and generalization capabilities, and can adapt to the defect judgment needs of LED products of different batches and models.

[0111] Preferably, see Figure 3 , the step of inputting the fused data into a pre-trained defect recognition model to obtain a defect recognition result includes:

[0112] Performing channel-by-channel feature extraction processing on the fused data to obtain a multimodal initial feature set;

[0113] Specifically, in the fused data, the original optical, electrical, and thermal features typically coexist in the form of multidimensional feature vectors or tensors. Channel-wise feature extraction refers to applying independent feature extraction operators or sub-networks to the feature channels of different modalities (such as brightness, temperature, and current waveform) to maintain the integrity of the information within the modality and obtain its key feature components, ultimately forming a multimodal initial feature set. The purpose of this step is to prevent inter-modal interference or information loss caused by mixing processing immediately after fusion. Through channel-wise extraction, the independent structural features and semantic information of each modality are retained, providing a stable foundation for subsequent inter-modal modeling. During implementation, three parallel sub-networks can be used: for example, a convolutional network extracts image texture and color features, an STFT or 1D CNN processes electrical signals, and a temperature gradient encoding network processes thermal maps. After extraction, a synchronized and aligned multimodal initial feature set is generated.

[0114] Based on the multimodal initial feature combination, inter-modal attention modeling and alignment processing are performed on each feature channel to obtain a fused feature tensor;

[0115] Specifically, inter-modal attention modeling is a strategy based on Transformer or multi-head attention mechanism, which is used to measure and dynamically adjust the relative importance of different modal channels in specific tasks or areas; through this modeling method, it is possible to automatically identify which type of information the current defect relies on more, such as a defect area that is not obvious in optical performance, but the thermal map and current signal are highly abnormal. The purpose of this step is to perform cross-modal saliency modeling and semantic alignment through the attention mechanism to obtain a fused feature tensor with high-dimensional structural expression capabilities, which contains the most representative features of each modality. During the implementation process, optical, electrical, and thermal features are used as multi-channel inputs, and interact through the attention mechanism to form a weighted fusion result. The attention mechanism assigns different weights to each channel according to the degree of coordination with other modalities, while maintaining spatial consistency through position encoding, and finally outputs a fused feature tensor with consistent structure and unified dimension.

[0116] According to the preset light color deviation template and defect instance database, the fused feature tensor is matched to obtain a target feature vector corresponding to the preset defect label;

[0117] Specifically, a preset light color deviation template and defect instance database are obtained as a knowledge base, and similarity matching is performed on the fused feature tensor to achieve automatic alignment and recognition of known defect patterns. The template library stores standard manifestation characteristics of typical defects, such as uneven brightness MULA distribution, flickering lines caused by current mutation, and hot spot expansion caused by abnormal thermal conductivity. Feature template matching is used to filter out the most discriminative regions or structures from the fused feature tensor, and extract target feature vectors that can be mapped one-to-one with historical defect labels. This vector provides high-confidence input for the subsequent classification model. In implementation, vector similarity calculations (such as cosine similarity and Euclidean distance) or methods based on KNN / template attention mechanisms can be used to compare the current input with labeled samples in the database in the feature space and select the closest feature response as the target vector. Deep features are linked with historical defect knowledge to realize a data-driven and knowledge-driven fusion reasoning mechanism, enhancing the system's robustness in identifying small-sample defects and complex problems.

[0118] According to the target feature vector, the fused feature tensor is input into the defect recognition model for forward recognition processing to obtain the defect recognition result, wherein the defect recognition result includes the defect type, defect location and defect level.

[0119] Specifically, the target feature vector and fused feature tensor are used as input and fed into the pre-trained defect recognition model for forward reasoning. The output includes a complete recognition result of the defect type (such as bright spots, dark spots, color deviation, flicker, etc.), defect location (such as coordinate area or pixel block) and defect level (such as minor, need to be reworked, scrapped), so as to achieve automatic, accurate and structured defect judgment, thereby replacing or assisting traditional manual visual inspection and local testing, and improving detection efficiency and consistency. The pre-trained defect recognition model adopts a multi-layer perceptron (MLP), graph neural network (GNN), or Transformer structure, combined with supervised learning methods for training; after input, the model calculates the confidence and position heat map of each category through forward propagation, and finally forms a structured output. By combining cross-modal deep features, comprehensive recognition of complex defects is achieved, and regional positioning capabilities are possessed, providing an accurate basis for subsequent process corrections.

[0120] According to the process parameter adjustment strategy corresponding to the defect identification result, the original process parameters corresponding to the LED display screen to be tested are adjusted to obtain the target process parameters;

[0121] Specifically, based on the defect type, location, and level information provided by the defect identification results, the system automatically matches the corresponding process parameter adjustment strategy and precisely adjusts the original process parameters of the LED display under test to generate optimized target process parameters. In this process, process parameters refer to key control variables that can be adjusted during display manufacturing or calibration, such as drive current, voltage waveform, grayscale response coefficient, PWM frequency, backlight zone brightness compensation, chip voltage offset, and thermal management parameters. According to the severity and distribution range of the defect level, the preset strategy library is called. For example, for defects in the uneven brightness level, the LED current in the corresponding area can be reduced or the regional grayscale calibration coefficient can be adjusted. The purpose is to build a defect-driven closed-loop feedback mechanism so that the detection results can not only be used for judgment, but also have a reverse effect on the production line, realizing rapid adjustment and repair suggestions based on the identification results; during the implementation process, the defect identification results are matched with the conditions in the strategy library, and after the adjustment plan is generated, it is transmitted to the driver chip, correction system or MES system through the control interface, which executes parameter writing or process modification, realizing a linkage closed loop of quality inspection and process control, greatly improving the response speed and automation level of the production line, effectively reducing manual intervention and reducing the rework rate. At the same time, it supports continuous optimization of the strategy library based on historical data, helping to achieve a more intelligent and efficient LED display quality control system.

[0122] Preferably, the process parameter adjustment strategy corresponding to the defect identification result is used to adjust the original process parameters corresponding to the LED display screen to be tested to obtain the target process parameters, including:

[0123] According to the defect type, a preset historical process parameter adjustment strategy library is searched and associated to obtain a parameter adjustment strategy set corresponding to the defect type;

[0124] Specifically, based on the identified defect type (such as uneven brightness, color shift, flicker, short circuit, etc.), the system intelligently matches and searches a preset library of historical process parameter adjustment strategies to extract a set of parameter adjustment strategies corresponding to the current defect. The strategy library records a large amount of process adjustment experience data associated with the defect type, covering adjustment targets (such as drive current, voltage bias, PWM frequency), applicable scope, execution conditions, and historical adjustment effects. For example, when a "uneven brightness - MULA" defect is identified, the strategy library retrieves a solution that "for MULA defects, the current in the area can be reduced by 10% and the grayscale coefficient α = 0.92"; if the defect is localized overheating, the strategy library returns parameter adjustment paths such as limiting peak current and adding thermal diffusion intervals. Using the defect identification result as a retrieval index, the parameter adjustment experience most relevant to this type of problem is activated, improving the adaptability and accuracy of the adjustment strategy. During implementation, semantic matching is performed between defect labels and index fields in the policy library. Keyword mapping, label encoding, consistent hashing, or matching methods based on vector similarity can be used for retrieval, ultimately returning a set of possible policies for subsequent module screening and execution. Introducing a structured knowledge base into the quality control system enables knowledge-driven integration of fault identification and adjustment decisions, improving system processing efficiency while significantly reducing reliance on human experience, helping to build an efficient, stable, and self-optimizing closed-loop production control framework.

[0125] Preferably, the process of performing an associated search on a preset historical process parameter adjustment strategy library according to the defect type to obtain a parameter adjustment strategy set corresponding to the defect type includes:

[0126] Performing semantic coding and standard classification processing on the defect type to obtain defect label information;

[0127] Specifically, by performing unified semantic coding and standard classification processing on the identified defect types, defect label information that can be used for structured retrieval is generated; semantic coding refers to converting defect types expressed in natural language (such as "bright spots", "dark spots", "edge color cast") into machine-readable labels in a unified format, and classifying defects hierarchically, such as establishing a label system based on functional dimensions (brightness class, chromaticity class, thermal anomaly class), severity level, spatial distribution and other dimensions. The purpose is to achieve standardization and labeling of defect identification results for subsequent use in knowledge base indexing, rule matching and data-driven automated decision processing. In specific implementation, through rule mapping tables, named entity recognition technology or Transformer-based semantic classification models, text or structured recognition results are mapped to standard labels and stored in the defect label pool as retrieval keys for subsequent operations, realizing a semantic transition from unstructured to structured, ensuring consistency and automation in subsequent process strategy retrieval and data analysis, and avoiding strategy matching errors due to inconsistent naming and ambiguous descriptions.

[0128] According to the defect label information, indexing is performed on a preset defect-process strategy mapping index table to obtain a candidate set of historical process parameter adjustment records;

[0129] Specifically, after the defect label encoding is completed, the label information will be used to access the constructed defect-process strategy mapping index table for rapid index retrieval to obtain a set of historical process parameter adjustment records related to the current defect label. When designing, the index table establishes a one-to-one or many-to-many relationship between common defect labels and parameter strategies that have been successfully adjusted, and records information such as the effects before and after the adjustment, applicable batches, and number of failures. Through the label-driven data indexing mechanism, past response strategies similar to the current problem can be quickly located, thereby improving the accuracy and response speed of defect adjustments. By establishing structures such as an inverted index table, a hash table, or a high-dimensional space index based on vectorized encoding (such as FAISS), multiple related parameter adjustment records are mapped out with the defect label as the key. The system can set a retrieval threshold to filter out low-relevance records to ensure that the candidate set has a high reference value.

[0130] Based on the candidate set of historical process parameter adjustment records, cluster analysis is performed on the process parameter change dimensions in the records to obtain a set of parameter adjustment paths that are highly correlated with the defect type;

[0131] Specifically, the index results are further mined for historical process parameter adjustment records. By performing statistical and cluster analysis on the process parameter change dimensions involved in these records, the set of parameter adjustment paths most closely associated with the current defect label is extracted. For example, combinations of chromaticity compensation matrix fine-tuning and grayscale correction parameter adjustments frequently appear in candidate records for color cast defects, constituting highly correlated adjustment paths. By identifying the inherent patterns between defect-parameter changes, sporadic or weakly correlated process interventions are filtered out, and the most representative optimization paths are output, providing data support for the final strategy ranking. K-means, DBSCAN, or hierarchical clustering algorithms can be used to map parameter change samples into feature space and cluster them based on dimensional change trends and impact outcomes. Clustering weights can also be incorporated into factors such as the change in ΔE before and after adjustment and the reduction in defect grade to enhance the practical guidance of path selection. This step establishes a significant channel between the defect and the adjustment path, eliminating non-causal factors and ensuring a more targeted and efficient output strategy set.

[0132] The parameter adjustment path set is weighted and sorted to obtain a priority-sorted defect process parameter adjustment strategy set.

[0133] Specifically, after obtaining a set of parameter adjustment paths highly relevant to the current defect, these paths are further weighted and ranked in a multi-dimensional manner based on metrics such as historical success rate, adjustment cost, response time, and scope of applicability, resulting in a prioritized set of defect process parameter adjustment strategies. Each path is assigned a comprehensive score during the ranking process, indicating its expected effectiveness and feasibility in resolving the current defect. The goal is to integrate historical experience with real-time judgment to generate a list of highly executable and prioritized strategies for automatic system invocation or manual confirmation by engineers. During implementation, a multi-factor scoring model can be constructed, comprehensively considering: historical adaptability (frequency of applicable defect labels), success rate (the magnitude of defect level reduction after execution), cost (whether it affects other areas), and real-time feasibility (whether dynamic delivery is supported). Ranking can be performed using a weighted linear model or a gradient-ranked tree-based ranking prediction, enabling rational quantification and prioritization of strategy decisions. This ensures that the system prioritizes adjustment paths with high effectiveness, low risk, and fast response when faced with complex defects, significantly improving repair efficiency and the system's closed-loop decision-making capabilities.

[0134] According to the defect location, performing spatial positioning mapping processing on the parameter adjustment strategy set to obtain a local process adjustment area corresponding to the defect location;

[0135] Specifically, based on the identified defect location information (usually expressed in the form of pixel coordinates, area block numbers or relative screen coordinates), the parameter adjustment strategy set generated in the previous link is spatially mapped to determine which physical area of ​​the LED display screen each parameter adjustment should act on, and finally obtain a local process adjustment area corresponding to the defect area. Among them, the parameter adjustment strategy set includes brightness gain adjustment, current compensation, local change of PWM frequency and other operations, and spatial positioning mapping requires that these strategies are no longer applied globally, but are accurately projected to the corresponding defect area through coordinate mapping, area masking and other methods. The purpose is to achieve accurate and localized repair of defects, avoid the waste of resources or quality degradation of other areas caused by traditional full-screen adjustment, and improve the flexibility and meticulousness of regulation. The implementation process includes: first, extracting the defective pixel blocks on the display screen through the location tags in the defect identification results (such as heat maps, border coordinates, and area IDs); then binding the parameter strategy (such as "reduce brightness by 5%") to the area, and limiting the scope of the strategy to this area through a spatial mask or control matrix; for micro-areas that require precise action, sub-pixel interpolation algorithms can also be used to ensure control accuracy, significantly improving the spatial resolution and control accuracy of the LED process control system, making parameter adjustments more targeted, avoiding over-correction, reducing energy consumption, and enhancing the overall consistency and detail quality of the screen. It is an important technical support point for moving towards high-end intelligent manufacturing.

[0136] Preferably, performing spatial positioning mapping processing on the parameter adjustment strategy set according to the defect location to obtain a local process adjustment area corresponding to the defect location includes:

[0137] According to the defect location, each pixel point in the display screen to be tested is partitioned to obtain the target pixel area where the defect is located;

[0138] Specifically, based on the identified defect location information, the entire LED display is divided into several logical areas or physical pixel blocks, and the target pixel area containing the defect is identified. Pixel partitioning processing refers to gridding the high-resolution screen into pixel blocks of fixed size (such as 8×8 or 16×16) for fine control and mapping. Based on the coordinate boundary or heat map distribution of the defect, it is determined in which pixel areas the defect falls, thereby forming a target area set. The macro defect location is refined into local areas with smaller control granularity, providing a basis for subsequent control unit positioning and accurate issuance of process strategies. In actual implementation, defective pixels are mapped to corresponding partition numbers through coordinate attribution determination (such as integer division operation, window sliding method); morphological processing (such as regional expansion) can also be used to expand the scope of influence to ensure the integrity of the repair edge.

[0139] According to the position tag corresponding to the target pixel area, obtaining a hardware control unit identifier corresponding to the position tag;

[0140] Specifically, the target pixel area is mapped to the actual hardware control unit, that is, the IC chip, PCB area, power channel or scanning unit that controls these areas is identified. The location label here is the pre-bound spatial identifier (such as block ID, grid number, IC number) in the aforementioned partition. The configuration table is searched through the label to obtain the hardware control unit identifier that controls the area, and accurate mapping from visual space coordinates to physical control units is achieved, so that process parameter adjustments can accurately act on the actual control module. The implementation method usually maps the area label to the hardware control unit by querying the spatial mapping table (determined by the screen layout and circuit wiring structure). In complex structures, auxiliary tables (such as scanning structure, matrix drive topology) can also be added to assist in determining the control path, realizing the logical alignment and linkage operation of image defect analysis and underlying circuit control, which is a key bridge from the detection system to the control execution system.

[0141] Filtering the parameter adjustment strategy set according to the hardware control unit identifier to obtain the local adjustment strategy subset;

[0142] Specifically, based on the hardware control unit identifier obtained in the previous step, a subset of local adjustment strategies related to the current control unit is screened out from the overall parameter adjustment strategy set; since the strategy set generated in the early stage may contain multiple regions or multiple types of adjustment items, targeted screening must be performed to avoid irrelevant regions from being misadjusted, to achieve local constraints and precision execution of strategies, and to ensure that each strategy only acts within the range of units that are actually controllable. In specific implementation, a "strategy-control unit mapping table" can be established or a Boolean filter can be used to compare the scope labels in all strategies with the control unit ID, and only strategies that intersect or are consistent with the current region are retained; for edge areas, a threshold can also be set to determine whether the strategy coverage meets the execution conditions. The technical effect is to ensure the uniqueness and precision of the strategy effect, avoid global misadjustment or waste of resources, and improve the stability and reliability of the process control system.

[0143] According to the effective area parameters in the local adjustment strategy subset, a display screen control interface or a driver chip control instruction is subjected to spatial offset mapping processing to obtain a local process adjustment parameter;

[0144] Specifically, the parameters of the action area in the local strategy are converted into actual executable control commands. The specific process includes the spatial offset and instruction mapping of the display control interface or the driver chip control register. The purpose is to complete the translation and mapping from the strategy semantic layer to the control signal layer to ensure that the strategy can be physically executed. The implementation methods include: using the regional control configuration table to find the control address corresponding to the region; using the programming interface provided by the driver chip (such as I 2 C, SPI, and PWM control protocols) are used to write strategy parameters into control registers. If the regional mapping is irregular, an interpolation algorithm or matrix conversion function can be added. The technical value of this step is that it realizes an efficient conversion link from process strategy to underlying hardware adjustment, establishing a complete closed loop of vision, decision-making, and control.

[0145] According to the local process adjustment parameters and in combination with the defect level information, an adjustable expansion process is performed on the boundary of the process adjustment region to obtain the local process adjustment region.

[0146] Specifically, considering the level information of the defect (such as mild, moderate, and severe), the boundary expansion processing is performed on the determined local adjustment area to ensure that the adjustment effect covers the defect edge or potential expansion area; the so-called adjustable expansion refers to dynamically controlling the expansion range according to the defect level, for example: mild defects expand outward by 1 pixel block, moderate defects expand by 2, and severe defects expand by more than 3, forming an adaptive compensation boundary, the purpose is to avoid the adjustment range being too narrow, resulting in residual defects on the edge or excessive local contrast, and to improve the consistency of the entire screen and the naturalness of the visual transition. In the implementation, morphological dilation operations, boundary fuzzy masks or boundary Gaussian expansion weights are usually used to achieve regional extension, and the distance field function from the defect center to the boundary is combined to generate a soft constraint control area. For extremely small defect areas, the minimum covering rectangle expansion strategy can also be used. This step improves the robustness and fault tolerance of the adjustment system. Through the level-driven expansion strategy, a more reliable and beautiful process repair effect can be achieved without significantly increasing resource overhead. It is an important part of achieving intelligent and delicate adjustment.

[0147] According to the defect level, performing amplitude correction processing on the adjustment strategy of the local process adjustment area to obtain an adjustment intensity parameter that matches the defect level;

[0148] Specifically, the process adjustment strategy established for the local area is corrected based on the identified defect level (such as minor, moderate, or severe), thereby generating adjustment intensity parameters that match the severity of the specific defect. The defect level is a standardized score calculated based on comprehensive indicators such as brightness deviation, color difference ΔE value, and temperature anomaly amplitude, reflecting the actual impact of the defect on display quality. The adjustment intensity parameters, such as brightness gain amplitude, current reduction ratio, and PWM frequency adjustment range, represent the intensity of strategy execution. The purpose is to dynamically adjust the intensity of strategy execution based on the actual impact of the defect, avoiding over-repair of minor defects or under-repair of severe defects, thereby achieving flexible adaptation and fine control of the adjustment strategy.

[0149] Preferably, performing amplitude correction processing on the adjustment strategy of the local process adjustment area according to the defect level to obtain an adjustment intensity parameter matching the defect level includes:

[0150] Performing a level quantization mapping process on the defect level to obtain a weight coefficient corresponding to the defect level;

[0151] Specifically, the defect level information is quantitatively mapped, that is, these level labels are converted into numerical weight coefficients so that they can be used as adjustment factors in the subsequent parameter adjustment process. The weight coefficient here can be regarded as the importance weight of the defect severity in the control strategy. The semantic defect level is converted into a numerical factor that can be used in the calculation, thereby supporting the subsequent strategy strength adjustment logic based on the mathematical model. During the implementation process, predefined mapping rules can be set, such as converting the defect level through a piecewise function, linear proportion or Sigmoid function. This conversion relationship can be obtained through actual experience annotation data or offline training model optimization, introducing a level-driven flexible adjustment mechanism for strategy parameter adjustment, providing a quantitative basis for the next step of precise adjustment strategy, and making the system have more delicate response capabilities.

[0152] Normalizing each adjustment parameter according to the initial adjustment strategy parameters in the local process adjustment area to obtain a reference value of the adjustable range;

[0153] Specifically, each initial strategy parameter in the local process area (such as brightness gain ratio, current adjustment amplitude, color temperature compensation value, etc.) is normalized, and the original parameters are uniformly mapped to the interval [0,1] or [-1,1] to obtain the reference adjustment value under each strategy dimension, that is, a standardized ratio indicator under the current maximum allowable adjustment intensity range. The purpose is to eliminate the differences in the dimensions and numerical scales of different strategy parameters through normalization operations, so as to effectively integrate and calculate with the weight coefficients. In the implementation method, the maximum and minimum values ​​are set for each strategy parameter (such as the brightness gain range of 0.8 to 1.2), and then normalization is performed. For bidirectional adjustment parameters, the symmetric normalization method can also be used to map the center value to 0, providing a unified and comparable standard reference scale for subsequent intensity adjustment calculations, making the calculation fusion between multi-dimensional parameters more reasonable, and avoiding adjustment imbalance or abnormality due to inconsistent dimensions.

[0154] Modifying the initial adjustment strategy parameters according to the weight coefficient and the reference value to obtain initial adjustment intensity parameters that match the defect level;

[0155] Specifically, the reference value obtained in the previous step is used to perform a fusion calculation with the aforementioned weight coefficient to correct the initial adjustment strategy and obtain an initial adjustment intensity parameter that matches the defect level; this parameter represents the actual degree of adjustment that should be applied for the current defect level within the theoretically allowed range; for example, if the reference adjustment value of a certain strategy is 0.8, and the defect weight coefficient is 0.6, then the final initial intensity is 0.8×0.6=0.48. The core purpose of this step is to achieve dynamic coupling between the adjustment strategy and the defect level, ensuring that the adjustment amplitude is neither excessive nor too small, and achieving the goal of accurate response. In actual implementation, different strategy dimensions can use linear fusion, weighted functions, or response curve modeling to generate intensity values. The system can also set an adjustment sensitivity factor and use different adjustment gain functions for different defect types (such as brightness type vs. hot spot type), establishing a flexible adjustment channel that drives parameter intensity with defect severity, improving strategy adaptability, avoiding the simplification of strategy intensity, and improving adjustment accuracy and safety.

[0156] According to the maximum allowable adjustment range corresponding to the initial adjustment strength parameter, an amplitude threshold limiting process is performed on it to obtain the adjustment strength parameter that meets the safety control boundary.

[0157] Specifically, the initial adjustment intensity parameter obtained above is compared with the maximum allowable adjustment range corresponding to the parameter. If it is found to exceed the safety range (for example, the current change exceeds the chip design limit), threshold clipping or saturation processing is performed to ensure that the final adjustment value is within the safe control boundary allowed by the hardware and process. By introducing a safety protection mechanism for adjustment intensity, it prevents the control parameter from exceeding the limit due to calculation errors or extreme defect levels, thereby protecting the hardware safety and adjustment stability of the display. The implementation process can automatically limit the intensity value based on pre-set safety upper and lower limits (for example, voltage adjustment must not exceed ±0.2V) using a clipping function, saturation function, or dynamic constraint mechanism.

[0158] The technical advantage of this step is that it adds a robustness and safety boundary barrier to the entire adjustment chain, ensuring that process adjustments do not cause additional load and damage to the LED driver unit, control circuit or optoelectronic characteristics while meeting the visual effect. This reflects the engineering feasibility and risk control capabilities of the system in industrial-level deployment.

[0159] The original process parameters are adjusted according to the adjustment intensity parameters and the local process adjustment area to obtain the target process parameters.

[0160] Specifically, the adjustment intensity parameters calculated in the previous step (such as brightness gain, current reduction ratio, and color temperature compensation value) are combined with the corresponding local process adjustment area to make targeted modifications to the original process parameters of the LED display (such as voltage, current, PWM duty cycle, lighting sequence, brightness correction matrix, etc.) to generate a new, executable target process parameter set. Among them, the "original process parameters" are the production control data initially generated by the equipment according to the default settings or basic calibration, while the "target process parameters" are new parameters obtained after optimization based on the detection results and control strategies, which are used to guide repairs. The purpose is to enable the production system to make accurate and quantitative process compensation or fine-tuning for the detected abnormal areas to repair local problems and restore the uniformity, performance stability and visual consistency of the entire screen. In actual applications, this process includes writing the brightness adjustment coefficient into the Gamma LUT (lookup table) of a specific pixel block, adjusting the driving voltage and current of the regional circuit, or adjusting the PWM signal frequency and duty cycle. For example, if an area has insufficient brightness and is assessed as having a slight defect, the system may adjust its brightness gain parameter from 1.00 to 1.07, and fine-tune the corresponding drive voltage from 2.8V to 2.9V to restore the target brightness level. The technical advantage of this step is that it builds a closed-loop control path driven by detection, regional orientation, and intensity adaptation, so that the process adjustment of the LED screen not only relies on manual experience or static configuration, but has the ability of self-learning and dynamic compensation, thereby improving the overall yield and process intelligence level.

[0161] According to the target process parameters, the process flow of the LED display screen to be tested is corrected to produce a qualified LED display screen that meets the quality requirements.

[0162] Specifically, after the target process parameters are generated, they are actually applied to the production process, and a new round of process correction processing based on the parameters is performed on the LED display screen to be tested to eliminate the original problems such as uneven brightness, electrical anomalies or thermal distribution deviations, and ultimately achieve the production of qualified LED display screens that meet the standards. The "process correction processing" here refers to the secondary processing or debugging of completed products in the production process, such as recalibrating brightness, adjusting circuit drive strategies, partially programming chip parameters, or re-executing some process links (such as reflow, relighting, etc.). The theoretical adjustment results are truly converted into physical repair behaviors and product quality improvement results, completing the last link from "intelligent detection" to "production execution", forming a truly closed-loop quality control system. Through the process control software, a new parameter table is written to the driver IC, the automated robotic arm is called to re-solder the defective area, the scanning order is changed to reduce the risk of heat concentration, or the cabinet is controlled to redisplay the standard test screen to verify the repair effect; the system will re-perform the "acquisition-identification-verification" process after the correction to confirm the repair effect, and build a dynamic production repair mechanism with self-feedback capability, which significantly improves the efficiency of repairing local defects, reduces the scrap rate and improves the yield rate.

[0163] Example 2

[0164] See also Figure 4 An embodiment of the present invention provides an LED display defect prediction and process adjustment system based on multimodal fusion, the system comprising at least one processor, at least one memory, and computer program instructions stored in the memory. When the computer program instructions are executed by the processor, the method described in Example 1 is implemented.

[0165] In addition, combined Figure 1 The LED display defect prediction and process adjustment method based on multimodal fusion described in the embodiment of the present invention can be implemented by an LED display defect prediction and process adjustment system based on multimodal fusion. Figure 4 A schematic diagram of the hardware structure of the LED display defect prediction and process adjustment system based on multimodal fusion provided by an embodiment of the present invention is shown.

[0166] The LED display defect prediction and process adjustment system based on multimodal fusion may include a processor and a memory storing computer program instructions.

[0167] Specifically, the processor may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits for implementing the embodiments of the present invention.

[0168] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0169] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated communication signals and carrier waves.

[0170] The processor reads and executes computer program instructions stored in the memory to implement any one of the LED display defect prediction and process adjustment methods based on multimodal fusion in the above embodiments.

[0171] In one example, the LED display defect prediction and process adjustment system based on multimodal fusion may further include a communication interface and a bus. Figure 4 As shown, the processor 401 , the memory 402 , and the communication interface 403 are connected via a bus 410 and communicate with each other.

[0172] The communication interface is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiments of the present invention.

[0173] The bus includes hardware, software, or both, coupling the components of the LED display defect prediction and process adjustment system based on multimodal fusion to each other. By way of example and not limitation, the bus may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industry standard architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnect, an industry standard architecture (ISA) bus, an infinite bandwidth interconnect, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus may include one or more buses. Although the embodiments of the present invention describe and illustrate specific buses, the present invention contemplates any suitable bus or interconnect.

[0174] In summary, the embodiments of the present invention provide a method and system for LED display defect prediction and process adjustment based on multimodal fusion.

[0175] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0176] It should also be noted that the exemplary embodiments described herein describe methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the steps described above. In other words, the steps may be performed in the order described in the embodiments, or in a different order, or several steps may be performed simultaneously.

[0177] The above description is only a specific embodiment of the present invention. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention.

Claims

1. A method for LED display defect prediction and process adjustment based on multimodal fusion, characterized in that: The method comprises: Obtain optical image data, electrical test data, and thermal infrared imaging data corresponding to the LED display to be tested; Performing multimodal data fusion processing on the optical image data, electrical test data, and thermal infrared imaging data to obtain fused data; Inputting the fused data into a pre-trained defect recognition model to obtain a defect recognition result; According to the process parameter adjustment strategy corresponding to the defect identification result, the original process parameters corresponding to the LED display screen to be tested are adjusted to obtain the target process parameters; According to the target process parameters, the process flow of the LED display screen to be tested is corrected to produce a qualified LED display screen that meets the quality requirements.

2. The method according to claim 1, characterized in that The step of obtaining optical image data, electrical test data, and thermal infrared image data corresponding to the LED display screen to be tested includes: Testing the LED display screen to be tested according to preset standard test conditions to obtain the real-time test status of the LED display screen to be tested; When the real-time test state reaches a preset test state, initial optical image data, initial electrical test data, and initial thermal infrared imaging data corresponding to the LED display screen to be tested are obtained; Performing time stamping and synchronization processing on the optical image initial data, the electrical test initial data, and the thermal infrared imaging initial data to obtain original multimodal data, wherein the original multimodal data includes original optical image data; Performing lens distortion correction and geometric transformation processing on the original optical image data to obtain standard optical image data; Performing affine transformation registration processing on the standard optical image data and the thermal infrared imaging initial data according to preset marking points to obtain spatially aligned multimodal image data; Performing drive signal period alignment processing on the multimodal image data and the electrical test initial data to obtain multimodal test data; The multimodal test data is normalized to obtain the optical image data, electrical test data and thermal infrared imaging data.

3. The method according to claim 1, characterized in that The multimodal data fusion processing is performed on the optical image data, the electrical test data and the thermal infrared imaging data to obtain the fused data, including: Performing feature extraction on the optical image data, electrical test data, and thermal infrared imaging data to obtain optical feature vectors, electrical feature vectors, and thermal feature vectors; Performing cross-modal correlation analysis on the optical eigenvector, the electrical eigenvector, and the thermal eigenvector to obtain a correlation weight matrix between the modes; performing feature fusion processing on the optical feature vector, the electrical feature vector, and the thermal feature vector according to the association weight matrix to obtain a primary fusion feature; Performing spatiotemporal consistency verification on the primary fusion features to obtain optimized multimodal fusion features; Performing dimension reduction and standardization processing on the multimodal fusion features to obtain the fusion data.

4. The method according to claim 3, characterized in that The extracting features of the optical image data, the electrical test data and the thermal infrared imaging data to obtain the optical feature vector, the electrical feature vector and the thermal feature vector respectively includes: Performing convolution feature extraction processing on the optical image data to obtain the optical feature vector including brightness distribution, chromaticity deviation and defect outline; Performing short-time Fourier transform processing on the electrical test data to obtain the electrical feature vector including harmonic components, current fluctuation characteristics, and voltage ripple characteristics; The thermal infrared imaging data is subjected to gradient field analysis processing to obtain the thermal characteristic vector including temperature gradient, hot spot distribution and heat conduction characteristics.

5. The method according to any one of claims 1 to 4, characterized in that Inputting the fused data into a pre-trained defect recognition model to obtain a defect recognition result includes: Performing channel-by-channel feature extraction processing on the fused data to obtain a multimodal initial feature set; Based on the multimodal initial feature combination, inter-modal attention modeling and alignment processing are performed on each feature channel to obtain a fused feature tensor; According to the preset light color deviation template and defect instance database, the fused feature tensor is matched to obtain a target feature vector corresponding to the preset defect label; According to the target feature vector, the fused feature tensor is input into the defect recognition model for forward recognition processing to obtain the defect recognition result, wherein the defect recognition result includes the defect type, defect location and defect level.

6. The method according to claim 5, characterized in that The original process parameters corresponding to the LED display screen to be tested are adjusted according to the process parameter adjustment strategy corresponding to the defect identification result to obtain target process parameters, including: According to the defect type, a preset historical process parameter adjustment strategy library is searched and associated to obtain a parameter adjustment strategy set corresponding to the defect type; According to the defect location, performing spatial positioning mapping processing on the parameter adjustment strategy set to obtain a local process adjustment area corresponding to the defect location; According to the defect level, performing amplitude correction processing on the adjustment strategy of the local process adjustment area to obtain an adjustment intensity parameter that matches the defect level; The original process parameters are adjusted according to the adjustment intensity parameters and the local process adjustment area to obtain the target process parameters.

7. The method according to claim 6, characterized in that According to the defect type, the preset historical process parameter adjustment strategy library is subjected to associated retrieval processing to obtain a parameter adjustment strategy set corresponding to the defect type, including: Performing semantic coding and standard classification processing on the defect type to obtain defect label information; According to the defect label information, a preset defect-process strategy mapping index table is indexed to obtain a candidate set of historical process parameter adjustment records; Based on the candidate set of historical process parameter adjustment records, cluster analysis is performed on the process parameter change dimensions in the records to obtain a set of parameter adjustment paths that are highly correlated with the defect type; The parameter adjustment path set is weighted and sorted to obtain a priority-sorted defect process parameter adjustment strategy set.

8. The method according to claim 6, characterized in that The performing of spatial positioning mapping processing on the parameter adjustment strategy set according to the defect position to obtain a local process adjustment area corresponding to the defect position includes: According to the defect location, each pixel point in the display screen to be tested is partitioned to obtain the target pixel area where the defect is located; According to the position tag corresponding to the target pixel area, obtaining a hardware control unit identifier corresponding to the position tag; Filtering the parameter adjustment strategy set according to the hardware control unit identifier to obtain the local adjustment strategy subset; According to the effective area parameters in the local adjustment strategy subset, a display screen control interface or a driver chip control instruction is subjected to spatial offset mapping processing to obtain a local process adjustment parameter; According to the local process adjustment parameters and in combination with the defect level information, an adjustable expansion process is performed on the boundary of the process adjustment region to obtain the local process adjustment region.

9. The method according to claim 6, characterized in that The step of performing amplitude correction processing on the adjustment strategy of the local process adjustment area according to the defect level to obtain an adjustment intensity parameter that matches the defect level includes: Performing a level quantization mapping process on the defect level to obtain a weight coefficient corresponding to the defect level; Normalizing each adjustment parameter according to the initial adjustment strategy parameters in the local process adjustment area to obtain a reference value of the adjustable range; Modifying the initial adjustment strategy parameters according to the weight coefficient and the reference value to obtain initial adjustment intensity parameters that match the defect level; According to the maximum allowable adjustment range corresponding to the initial adjustment strength parameter, an amplitude threshold limiting process is performed on it to obtain the adjustment strength parameter that meets the safety control boundary.

10. A LED display defect prediction and process adjustment system based on multimodal fusion, the system comprising at least one processor, at least one memory, and computer program instructions stored in the memory, wherein when the computer program instructions are executed by the processor, the method according to any one of claims 1 to 9 is implemented.

Images