LED display screen fault detection method and system

Abstract

The invention discloses an LED display screen fault detection method and system. The method comprises the following steps: collecting a display image of an LED display screen; preprocessing the display image; inputting the pre-processed display image into a pre-trained convolution gating circulation deep learning model to detect whether the LED display screen has a fault or not; if a fault is detected, classifying and positioning the fault; and performing hybrid effect analysis to evaluate the influence of environmental factors and operating parameters on the fault. The invention also provides an LED display screen fault detection system for realizing the method. According to the method and the system, efficient and accurate detection of faults of the LED display screen can be realized, deeper fault diagnosis is provided through hybrid effect analysis, and the maintenance efficiency and the reliability of the LED display screen are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of display technology and artificial intelligence, and specifically relates to a method and system for real-time fault detection and diagnosis of an LED display screen using a deep learning model. Background Art

[0002] LED displays are widely used in advertising, sports stadiums, traffic signs, etc. due to their advantages such as high brightness, low power consumption and long life. However, as the scale of LED displays continues to expand and the application scenarios become increasingly complex, their fault detection and maintenance have become an important technical challenge.

[0003] At present, fault detection of LED display screens mainly relies on manual inspection or simple hardware monitoring. For example, faults are identified by regular manual observation of display effects, or by monitoring parameters such as drive current and temperature to determine whether there are abnormalities. Although these methods are simple and direct, they have problems such as low efficiency and high false alarm rate.

[0004] In recent years, with the development of computer vision and deep learning technology, some researchers have tried to apply image processing and machine learning methods to LED display screen fault detection. For example, some studies have proposed using convolutional neural networks to analyze LED display screen images to identify faults such as pixel failure. This type of method has improved the automation and accuracy of fault detection to a certain extent.

[0005] However, the existing LED display fault detection technology still has the following problems:

[0006] 1. Low detection efficiency: Existing methods make it difficult to achieve real-time and comprehensive monitoring of large-scale LED display screens.

[0007] 2. Insufficient detection accuracy: It is difficult to accurately identify and classify various fault types, especially early fault signs.

[0008] 3. Weak positioning capability: Unable to accurately locate the fault area, increasing the difficulty and time of maintenance.

[0009] 4. Lack of consideration of environmental factors: The impact of environmental factors and operating parameters on faults is not fully considered, resulting in false alarms and missed alarms. Summary of the invention

[0010] In view of this, the present application provides a method and system for detecting faults in an LED display screen, which solves the problems of low efficiency, insufficient accuracy, weak positioning capability and lack of consideration of environmental factors in the prior art of fault detection in an LED display screen.

[0011] The present application provides a method for detecting LED display screen faults, including:

[0012] Collect the display image of the LED display;

[0013] Preprocessing the display image;

[0014] Inputting the preprocessed display image into a pre-trained convolutional gated recurrent deep learning model to detect whether the LED display screen has a fault;

[0015] If a fault is detected, classifying and locating the fault;

[0016] Confounding effect analysis was performed to evaluate the impact of environmental factors and operating parameters on the failure.

[0017] The preprocessing of the display image includes: performing denoising on the acquired display image; performing enhancement on the denoised display image; and performing standardization on the enhanced display image.

[0018] The convolutional gated recurrent deep learning model includes: a convolutional layer group for extracting the spatial features of the displayed image; a gated recurrent unit layer for capturing temporal features; and a fully connected layer for integrating spatial and temporal features; wherein the convolutional layer group includes a plurality of convolutional blocks, each convolutional block includes a 2D convolutional layer, a batch normalization layer, a ReLU activation function layer, and a maximum pooling layer, and the 2D convolutional layer uses convolution kernels of different sizes to extract multi-scale features.

[0019] Before the preprocessed display image is input into the pre-trained convolutional gated recurrent deep learning model, it also includes: training the convolutional gated recurrent deep learning model, including: acquiring training data, performing data enhancement processing on the training data, including random rotation, scaling, brightness and contrast adjustment; using the ResNet50 model pre-trained on the ImageNet data set as a feature extractor to extract the initial features of the LED display image; inputting the extracted initial features into the convolutional gated recurrent deep learning model, and optimizing the three subtasks of fault detection, classification and positioning at the same time.

[0020] The method also includes: adding L1 and L2 regularization terms in the loss function to suppress the complexity of the convolutional gated loop deep learning model; using the ReLU6 activation function to limit the activation value to the range of [0,6] to achieve sparse activation; using the initial module inception, applying 1x1, 3x3 and 5x5 convolution kernels in parallel to achieve multi-scale feature extraction; introducing an attention mechanism to dynamically adjust the importance of features of different scales according to the input to achieve dynamic weight adjustment.

[0021] The classification and location of the faults include: classifying the detected faults, including at least one or more combinations of pixel failure, color anomaly, uneven brightness, flicker, image tearing and module failure; using attention mechanism and heat map technology to accurately locate the position and range of the fault area; quantitatively evaluating the severity of the fault based on the fault type, area and duration; and dynamically adjusting the sensitivity and threshold of the fault detection based on the fault detection results.

[0022] The confounding effect analysis includes: performing variable confounding detection to analyze the confounding relationship between environmental factors and operating parameters and the occurrence of faults; using causal inference technology to distinguish and quantify the impact of observable variables and potential unobserved variables on faults; calculating the relative strength of confounding deviations between different sets of variables to identify key influencing factors; optimizing the operating environment and parameter settings of the LED display based on the results of the confounding effect analysis; wherein the variable confounding detection includes calculating the Pearson correlation coefficient matrix between variables, calculating the variance inflation factor for each independent variable, and performing partial correlation analysis to identify direct and indirect relationships.

[0023] The method of distinguishing and quantifying the influence of observable variables and potential unobserved variables on failures is implemented by using a structural equation model, including: constructing a structural equation model specification containing latent variables, defining the relationship between observed variables and latent variables; estimating the parameters of the structural equation model using the maximum likelihood estimation method; evaluating the goodness of fit of the structural equation model, including the chi-square test, the comparative fit index CFI and the root mean square error approximation RMSEA; analyzing the path coefficient and effect size, and quantifying the influence of observed variables and latent variables on failures; the method of calculating the relative strength of confounding bias between different sets of variables is implemented by using a partial R-square analysis, including: constructing a complete linear regression model containing all predictor variables; calculating the R-square value of the complete linear regression model; removing each predictor variable one by one to construct a reduced model; calculating the R-square value of each reduced model; calculating the partial R-square value of each predictor variable, including the difference between the full model R-square and the corresponding reduced model R-square; comparing the partial R-square values ​​of different predictor variables to determine the unique contribution of each variable to the frequency of failure.

[0024] The method also includes: using the LSHADE algorithm with restart and local search mechanism to optimize model hyperparameters and system configuration, and the specific steps include: initializing the population and algorithm parameters; evaluating the objective function value for each individual; applying differential evolution operation to generate test vectors; adaptively adjusting the scaling factor and crossover probability according to the success rate history; performing local search to improve the current optimal solution; updating the population and archiving; checking the restart condition and reinitializing the population when necessary; repeating the above steps until the termination condition is reached.

[0025] The present application also provides a LED display screen fault detection system, including:

[0026] An acquisition module is used to acquire the display image of the LED display screen;

[0027] A preprocessing module, used for preprocessing the display image;

[0028] A detection module, used for inputting the preprocessed display image into a pre-trained convolutional gated recurrent deep learning model to detect whether the LED display screen has a fault;

[0029] A classification and location module, for classifying and locating the fault if a fault is detected;

[0030] Analysis module for performing confounding effect analysis to evaluate the impact of environmental factors and operating parameters on faults.

[0031] The present application also provides a computer device, the computer device comprising:

[0032] at least one processor; and,

[0033] a memory communicatively connected to the at least one processor; wherein,

[0034] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned LED display screen fault detection method.

[0035] The embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the above-mentioned LED display screen fault detection method.

[0036] The embodiment of the present application also provides a computer program product, including computer instructions, which implement the steps of the above-mentioned LED display screen fault detection method when executed by a processor.

[0037] This application has the following technical effects:

[0038] This application uses a convolutional gated recurrent deep learning model combined with multiple optimization techniques to achieve efficient and accurate detection of LED display screen faults. The model can simultaneously capture the spatial and temporal features of the image, greatly improving the accuracy and real-time performance of fault detection.

[0039] This application introduces confounding effect analysis, which deeply analyzes the impact of environmental factors and operating parameters on faults through variable confounding detection, causal inference and bias strength assessment. This innovation makes fault diagnosis more comprehensive and accurate, helping to reduce false positives and false negatives.

[0040] This application uses attention mechanism and heat map technology to achieve accurate fault location and greatly improve maintenance efficiency. At the same time, by dynamically adjusting the detection sensitivity and threshold, this method can adapt to different operating environments and conditions and maintain long-term detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 A flow chart of a LED display screen fault detection method provided by an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of the structure of a convolutional gated recurrent deep learning model provided by an embodiment of the present invention;

[0044] Figure 3 A flow chart of confounding effect analysis provided by an embodiment of the present invention;

[0045] Figure 4 A thermal diagram of LED display screen fault detection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] Example 1

[0047] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0048] Embodiment 1

[0049] The core of the present invention is to provide an efficient and accurate LED display screen fault detection method and system. The present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] Please refer to Figure 1 , which shows a flow chart of a LED display screen fault detection method provided by an embodiment of the present invention. The method comprises the following steps:

[0051] S1: Collect the display image of the LED display screen.

[0052] The system uses a high-resolution camera to capture the display image of the LED display in real time. The resolution and frame rate of the camera should be high enough to capture subtle changes on the LED display. For example, a camera with 4K resolution (3840×2160 pixels) and 60 frames per second can be used. In order to cover the entire LED display, multiple cameras are required to work together to form a camera array.

[0053] S2: Preprocess the display image.

[0054] This step includes the following sub-steps:

[0055] S2.1: De-noising the acquired display image.

[0056] Denoising is aimed at reducing random noise in the image and improving image quality. This embodiment adopts the Non-Local Means (NLM) denoising algorithm. The basic idea of ​​the NLM algorithm is to use the redundant information in the image and perform denoising by searching for similar blocks in the search window. The specific implementation is as follows:

[0057] ```Python

[0058] import cv2

[0059] import numpy as np

[0060] def nlm_denoising(image,h=10,templateWindowSize=7,searchWindowSize=21):

[0061] return cv2.fastNlMeansDenoisingColored(image,None,h,h,templateWindowSize,searchWindowSize)

[0062] #Usage examples

[0063] denoised_image=nlm_denoising(original_image)

[0064] ```

[0065] Among them, the h parameter controls the filtering strength, templateWindowSize is the template window size, and searchWindowSize is the search window size. These parameters can be adjusted according to the specific image characteristics and noise level.

[0066] S2.2: Perform enhancement processing on the denoised display image.

[0067] The purpose of image enhancement is to improve the contrast and clarity of the image and make the fault features more obvious. This embodiment uses the Adaptive Histogram Equalization (AHE) algorithm for image enhancement. The advantage of the AHE algorithm is that it can locally improve the contrast of the image and avoid the problem of over-enhancement caused by global histogram equalization. The specific implementation is as follows:

[0068] ```Python

[0069] import cv2

[0070] def enhance_image(image):

[0071] lab=cv2.cvtColor(image,cv2.COLOR_BGR2LAB)

[0072] l,a,b=cv2.split(lab)

[0073] clahe=cv2.createCLAHE(clipLimit=2.0,tileGridSize=(8,8))

[0074] cl = clahe.apply(l)

[0075] limg = cv2.merge((cl,a,b))

[0076] enhanced=cv2.cvtColor(limg,cv2.COLOR_LAB2BGR)

[0077] return enhanced

[0078] #Usage examples

[0079] enhanced_image=enhance_image(denoised_image)

[0080] ```

[0081] In this implementation, the embodiment of the present application first converts the image from the BGR color space to the LAB color space, and then applies the CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm only to the L channel (brightness channel). This avoids changing the color information of the image and only enhances the brightness contrast.

[0082] S2.3: Perform standardization processing on the enhanced display image.

[0083] The purpose of normalization is to scale the image pixel values ​​to a fixed range, usually [0, 1] or [-1, 1]. This helps improve the training efficiency and generalization ability of the deep learning model. This embodiment adopts the MinMax normalization method, which is specifically implemented as follows:

[0084] ```Python

[0085] def normalize_image(image):

[0086] normalized=(image-np.min(image)) / (np.max(image)-np.min(image))

[0087] return normalized

[0088] #Usage examples

[0089] normalized_image=normalize_image(enhanced_image)

[0090] ```

[0091] Through the above preprocessing steps, the embodiment of the present application obtains a denoised, enhanced and standardized LED display screen image, providing high-quality input data for subsequent fault detection.

[0092] S3: Input the preprocessed display image into a pre-trained convolutional gated recurrent deep learning model to detect whether there is a fault in the LED display.

[0093] Please refer to Figure 2 , which shows a schematic diagram of the structure of a convolutional gated recurrent deep learning model (CGRDLM) provided by an embodiment of the present invention. The model combines a convolutional neural network (CNN) and a gated recurrent unit (GRU) to simultaneously capture the spatial and temporal features of an image. The specific structure and training process of the model are as follows:

[0094] S3.1: Model structure

[0095] The CGRDLM model includes the following main components:

[0096] 1. Convolutional layer group:

[0097] It contains multiple convolution blocks, each of which consists of a 2D convolution layer, a batch normalization layer, a ReLU activation function layer, and a maximum pooling layer.

[0098] The 2D convolutional layer uses convolution kernels of different sizes (such as 3x3, 5x5, 7x7) to extract multi-scale features.

[0099] Batch Normalization layers are used to speed up the training process and improve model stability.

[0100] The ReLU activation function introduces nonlinearity and enhances the expressiveness of the model.

[0101] The maximum pooling layer is used to reduce the size of the feature map and extract salient features.

[0102] 2. Gated Recurrent Unit (GRU) layer:

[0103] Used to capture timing features and process the relationship between consecutive frames.

[0104] Includes update gate and reset gate to control the flow and forgetting of information.

[0105] 3. Fully connected layer:

[0106] The spatial and temporal features are integrated for final fault detection and classification.

[0107] 4. Output layer:

[0108] Use the Softmax activation function to output the probability distribution of various fault types.

[0109] 5. Low entropy design:

[0110] L1 / L2 regularization: Add L1 and L2 regularization terms to the loss function to suppress model complexity.

[0111] For example: loss=cross_entropy+λ1*L1_norm+λ2*L2_norm

[0112] Where λ1 and λ2 are adjustable hyperparameters.

[0113] Sparse activation: Use the ReLU6 activation function after the convolutional layer to limit the range of activation values ​​to [0, 6].

[0114] ReLU6(x)=min(max(0,x),6)

[0115] 6. Heterogeneous Spectral Dynamics:

[0116] Multi-scale convolution: Using the Inception module, convolution kernels of different sizes are applied in parallel.

[0117] For example:

[0118] 1x1 convolution: capturing point-level features

[0119] 3x3 convolution: capturing local features

[0120] 5x5 convolution: captures features in a wider range

[0121] Dynamic weight adjustment: Introduce the attention mechanism to dynamically adjust the importance of features of different scales according to the input.

[0122] attention_weights=softmax(FC(concat([1x1_conv,3x3_conv,5x5_conv])))

[0123] output=attention_weights[0]*1x1_conv+attention_weights[1]*3x3_conv+attention_weights[2]*5x5_conv

[0124] The following is an example of the Python code implementation of the model:

[0125] ```Python

[0126] import tensorflow as tf from tensorflow.keras import layers,models

[0127] def create_cgrdlm_model(input_shape,num_classes):

[0128] inputs=layers.Input(shape=input_shape)

[0129] #Convolutional layer group

[0130] x=layers.Conv2D(32,(3,3),activation='relu',padding='same')(inputs)x=layers.BatchNormalization()(x)

[0131] x=layers.MaxPooling2D((2,2))(x)

[0132] x=layers.Conv2D(64,(3,3),activation='relu',padding='same')(x)x=layers.BatchNormalization()(x)

[0133] x=layers.MaxPooling2D((2,2))(x)

[0134] x=layers.Conv2D(128,(3,3),activation='relu',padding='same')(x)x=layers.BatchNormalization()(x)

[0135] x=layers.MaxPooling2D((2,2))(x)

[0136] #Reshape the convolution output to fit the GRU layer

[0137] x=layers.Reshape((-1,x.shape[-1]))(x)

[0138] #GRU layer

[0139] x=layers.GRU(256,return_sequences=True)(x)

[0140] x = layers.GRU(128)(x)

[0141] #Fully connected layer

[0142] x=layers.Dense(128,activation='relu')(x)

[0143] x = layers.Dropout(0.5)(x)

[0144] #Output layer

[0145] outputs=layers.Dense(num_classes,activation='softmax')(x)

[0146] model=models.Model(inputs=inputs,outputs=outputs)

[0147] return model

[0148] #Usage examples

[0149] input_shape = (224, 224, 3) # Assume the input image size is 224x224, 3 color channels num_classes = 6 # Assume there are 6 types of faults

[0150] model=create_cgrdlm_model(input_shape,num_classes)

[0151] ```

[0152] S3.2: Model training

[0153] The model training process includes the following steps:

[0154] 1. Data Augmentation:

[0155] Use various image transformation techniques to expand the training data set and improve the generalization ability of the model. The specific implementation is as follows:

[0156] ```Python

[0157] from tensorflow.keras.preprocessing.image import ImageDataGeneratordef create_data_generator():

[0158] return ImageDataGenerator(

[0159] rotation_range=20,

[0160] width_shift_range=0.2,

[0161] height_shift_range=0.2,

[0162] shear_range=0.2,

[0163] zoom_range=0.2,

[0164] horizontal_flip=True,

[0165] fill_mode = 'nearest' )

[0167] #Usage examples

[0168] train_datagen=create_data_generator()

[0169] train_generator=train_datagen.flow(x_train,y_train,batch_size=32)

[0170] ```

[0171] 2. Transfer Learning:

[0172] The ResNet50 model pre-trained on the ImageNet dataset is used for feature extraction to accelerate model convergence.

[0173] The specific implementation is as follows:

[0174] ```Python

[0175] from tensorflow.keras.applications import ResNet50

[0176] from tensorflow.keras.models import Model

[0177] def create_feature_extractor(input_shape):

[0178] base_model=ResNet50(weights='imagenet',include_top=False,input_shape=input_shape)

[0179] for layer in base_model.layers:

[0180] layer.trainable = False

[0181] return base_model

[0182] #Usage examples

[0183] feature_extractor=create_feature_extractor(input_shape)

[0184] features=feature_extractor.predict(normalized_image)

[0185] ```

[0186] 3. Multi-task learning:

[0187] At the same time, the three subtasks of fault detection, classification and location are optimized to improve the overall performance of the model. The specific implementation is as follows:

[0188] ```Python

[0189] def multi_task_loss(y_true,y_pred):

[0190] detection_loss=tf.keras.losses.binary_crossentropy(y_true[0],y_pred[0])

[0191] classification_loss=tf.keras.losses.categorical_crossentropy(y_true[1],y_pred[1])

[0192] localization_loss=tf.keras.losses.mean_squared_error(y_true[2],y_pred[2])

[0193] return detection_loss+classification_loss+localization_loss

[0194] #Usage examples

[0195] model.compile(optimizer='adam',loss=multi_task_loss,metrics=['accuracy'])

[0196] ```

[0197] Exemplarily, data augmentation includes:

[0198] Random Rotation: Randomly rotate the image in the range [-10°, 10°].

[0199] Random Scale: Randomly scale the image in the range [0.9, 1.1].

[0200] Random brightness and contrast adjustments: simulate different lighting conditions.

[0201] Add Gaussian noise: Enhance the model’s robustness to noise.

[0202] In addition, in transfer learning, the weights of the first few layers of ResNet50 are frozen, and only the subsequent layers are fine-tuned.

[0203] For example:

[0204] ```Python

[0205] base_model=ResNet50(weights='imagenet',include_top=False)

[0206] for layer in base_model.layers[:100]:

[0207] layer.trainable = False

[0208] x = base_model.output

[0209] x=GlobalAveragePooling2D()(x)

[0210] x=Dense(1024,activation='relu')(x)

[0211] output=Dense(num_classes,activation='softmax')(x)

[0212] model=Model(inputs=base_model.input,outputs=output)

[0213] ```

[0214] For example, multi-task learning requires optimizing three related tasks simultaneously:

[0215] 1. Fault detection (binary classification): Determine whether a fault exists.

[0216] 2. Fault classification (multi-classification): Identify the specific fault type.

[0217] 3. Fault location (regression): predict the coordinates and size of the fault area.

[0218] Loss function design:

[0219] ```Python

[0220] total_loss=w1*binary_crossentropy(y_true_detection,y_pred_detection)+

[0221] w2*categorical_crossentropy(y_true_classification,y_pred_classification)+

[0222] w3*mean_squared_error(y_true_localization,y_pred_localization)

[0223] ```

[0224] Among them, w1, w2, w3 are the weights of each task, which can be tuned through the validation set.

[0225] Training strategy:

[0226] The Adam optimizer is used and the initial learning rate is set to 0.001.

[0227] The learning rate decay strategy is used, decreasing by 10% every 10 epochs.

[0228] An early stopping strategy is applied, and training is stopped when the validation set loss does not improve for 5 consecutive epochs.

[0229] Use mini-batch gradient descent with a batch size of 32.

[0230] 4. Low entropy design:

[0231] By introducing regularization techniques and sparse activation functions, the information entropy of the model is reduced and the generalization ability and robustness of the model are improved.

[0232] Low entropy design is a method that aims to improve the generalization ability and robustness of a model. In information theory, entropy is a measure of the uncertainty of a system. In deep learning, low entropy design means reducing the complexity of the model through various technical means, reducing the risk of overfitting, and thus improving the performance of the model on unseen data.

[0233] In the present invention, low entropy design is mainly achieved in the following two ways:

[0234] a) Regularization Techniques:

[0235] Regularization is a method of limiting the complexity of a model by adding a penalty term to the loss function. The present invention uses a combination of L1 and L2 regularization, also known as elastic net regularization.

[0236] L1 regularization (Lasso) adds an absolute value penalty term, which tends to produce sparse solutions, even if the weights of some features become zero. This helps in feature selection and removes unimportant features.

[0237] L2 regularization (Ridge) adds a square penalty term, tending to make the weights of all features small but non-zero. This helps deal with multicollinearity problems.

[0238] By combining these two regularization methods, the embodiments of the present application can achieve a balance between feature selection and weight decay, effectively reducing the complexity of the model.

[0239] For example, when detecting color anomalies in LED displays, L1 regularization may reduce the weights of some less important color channels to zero, while L2 regularization smoothly reduces the weights of all color channels. This combination can help the model focus on the most critical color features while maintaining sensitivity to all color information.

[0240] b) Sparse activation function:

[0241] The present invention uses ReLU6 (Rectified Linear Unit 6) as the activation function. The definition of ReLU6 is as follows:

[0242] f(x)=min(max(0,x),6)

[0243] ReLU6 limits the activation value to the range [0,6]. This design has several advantages:

[0244] 1. Sparsity: Negative inputs produce zero outputs, resulting in sparse network activations, which helps reduce overfitting.

[0245] 2. Nonlinearity: The nonlinear characteristics of ReLU are retained, allowing the model to learn complex nonlinear relationships.

[0246] 3. Boundedness: The upper limit of 6 can prevent the activation value from being too large, which helps numerical stability.

[0247] In LED display fault detection, ReLU6 can help the model better process brightness information. For example, when detecting uneven brightness, ReLU6 can effectively capture subtle changes in brightness while preventing extremely high brightness values ​​from having too much impact on the model.

[0248] The specific implementation is as follows:

[0249] ```Python

[0250] from tensorflow.keras import regularizers

[0251] def add_regularization(model,weight_decay=1e-4):

[0252] for layer in model.layers:

[0253] if isinstance(layer,tf.keras.layers.Conv2D)or isinstance(layer,tf.keras.layers.Dense):

[0254] layer.add_loss(lambda:

[0255] tf.keras.regularizers.l2(weight_decay)(layer.kernel))

[0256] if hasattr(layer,'bias_regularizer')and layer.use_bias:

[0257] layer.add_loss(lambda:

[0258] tf.keras.regularizers.l2(weight_decay)(layer.bias))

[0259] return model

[0260] def relu6(x):

[0261] return tf.keras.backend.relu(x,max_value=6)

[0262] #Usage examples

[0263] model=add_regularization(model)

[0264] model.add(tf.keras.layers.Activation(relu6))

[0265] ```

[0266] 5. Heterogeneous spectral dynamics:

[0267] The multi-scale convolution kernel and dynamic weight adjustment mechanism are used to enable the model to capture fault characteristics of different frequencies and scales.

[0268] Heterogeneous spectral dynamics is a technique designed to capture fault features at different frequencies and scales. In LED display fault detection, different types of faults may manifest at different spatial scales and temporal frequencies. For example, a single pixel fault may be a small-scale high-frequency feature, while a whole module fault may be a large-scale low-frequency feature.

[0269] The present invention realizes heterogeneous spectrum dynamics in the following two ways:

[0270] a) Multi-scale convolution kernel:

[0271] Adopting the idea of ​​the initial Inception module, different sizes of convolution kernels (such as 1x1, 3x3, 5x5) are used in parallel. This allows the model to extract features at different receptive field sizes at the same time.

[0272] For example, when detecting LED display failure:

[0273] 1x1 convolution can capture single pixel level failures such as dead pixels.

[0274] 3x3 convolution can detect small-scale fault patterns, such as dark or bright areas consisting of a few pixels.

[0275] The 5x5 convolution can identify faults on a larger scale, such as anomalies in an entire row or column of LEDs.

[0276] By combining these features at different scales, the model can fully understand the status of the LED display and perform fault detection from the micro to the macro level.

[0277] b) Dynamic weight adjustment mechanism:

[0278] The attention mechanism is introduced to dynamically adjust the importance of features of different scales according to the input. This allows the model to adaptively focus on the most relevant feature scales.

[0279] In practice, dynamic weight adjustment works as follows:

[0280] 1. Calculate a global average pooling value for the convolution output of each scale to obtain the global features of each scale.

[0281] 2. Concatenate these global features together and calculate the weights of each scale through a small neural network (usually a fully connected layer plus softmax activation).

[0282] 3. Use these weights to perform weighted summation of features at different scales to obtain the final feature representation.

[0283] For example, when a large area of ​​brightness anomaly is detected, the model may automatically increase the weight of the 5x5 convolution kernel to better capture this large-scale fault. When encountering subtle color deviations, the model may increase the weight of the 1x1 and 3x3 convolution kernels to capture finer color changes.

[0284] Through this dynamic adjustment mechanism, the model can adaptively adjust its receptive field according to different types of LED display faults, thereby improving the accuracy and flexibility of fault detection.

[0285] This heterogeneous spectral dynamics design enables the model of the present application embodiment to simultaneously process fault features of different scales and frequencies, greatly improving the comprehensiveness and accuracy of LED display screen fault detection. Whether it is a small single point fault or a large-scale systemic problem, it can be effectively captured and identified.

[0286] The specific implementation is as follows:

[0287] ```Python

[0288] def inception_module(x,filters):

[0289] #1x1 convolution

[0290] conv1x1=tf.keras.layers.Conv2D(filters,(1,1),padding='same',activation='relu')(x)

[0291] #3x3 convolution

[0292] conv3x3=tf.keras.layers.Conv2D(filters,(3,3),padding='same',activation='relu')(x)

[0293] #5x5 convolution

[0294] conv5x5=tf.keras.layers.Conv2D(filters,(5,5),padding='same',activation='relu')(x)

[0295] #Attention Mechanism

[0296] concat=tf.keras.layers.Concatenate()([conv1x1,conv3x3,conv5x5])

[0297] attention=tf.keras.layers.Dense(3,activation='softmax')(tf.keras.layers.GlobalAveragePooling2D()(concat))

[0298] #Dynamic weight adjustment

[0299] output=tf.keras.layers.Add()([

[0300] tf.keras.layers.Multiply()([conv1x1,tf.keras.layers.Reshape((1,1,1))(attention[:,0])]),

[0301] tf.keras.layers.Multiply()([conv3x3,tf.keras.layers.Reshape((1,1,1))(attention[:,1])]),

[0302] tf.keras.layers.Multiply()([conv5x5,tf.keras.layers.Reshape((1,1,1))(attention[:,2])]) ])

[0304] return output

[0305] #Usage examples

[0306] x=inception_module(input_layer,64)

[0307] ```

[0308] Through the above training strategy, the embodiment of the present application can obtain an efficient and accurate convolutional gated recurrent deep learning model for LED display screen fault detection.

[0309] S4: If a fault is detected, the fault is classified and located.

[0310] This step includes the following sub-steps:

[0311] S4.1: Fault classification

[0312] Classify detected faults, including pixel failure, color anomaly, uneven brightness, flicker, image tearing, and module failure. The specific implementation is as follows:

[0313] ```Python

[0314] def classify_fault(model_output):

[0315] fault_types = ['Pixel failure', 'Color abnormality', 'Uneven brightness', 'Flickering', 'Image tearing', 'Module failure']

[0316] predicted_class=np.argmax(model_output)

[0317] confidence=model_output[predicted_class]

[0318] return fault_types[predicted_class],confidence

[0319] #Usage examples

[0320] fault_type,confidence=classify_fault(model.predict(preprocessed_image))

[0321] ```

[0322] S4.2: Fault location

[0323] Use the attention mechanism and heat map technology to accurately locate the location and range of the fault area. The specific implementation is as follows:

[0324] ```Python

[0325] import tensorflow as tf

[0326] import cv2

[0327] import numpy as np

[0328] def generate_heatmap(model,img_array,last_conv_layer_name):

[0329] #Create a model that only outputs the last convolutional layer

[0330] grad_model=tf.keras.models.Model(

[0331] [model.inputs],[model.get_layer(last_conv_layer_name).output,model.output] )

[0333] #Calculate gradients

[0334] with tf.GradientTape()as tape:

[0335] conv_output,predictions=grad_model(img_array)

[0336] loss=predictions[:,np.argmax(predictions[0])]

[0337] #Extract relevant convolution output

[0338] output = conv_output[0]

[0339] grads=tape.gradient(loss,conv_output)[0]

[0340] #Global average pooling

[0341] weights=tf.reduce_mean(grads,axis=(0,1))

[0342] #Generate class activation heat map

[0343] cam = np.dot(output,weights)

[0344] cam=cv2.resize(cam,(img_array.shape[1],img_array.shape[2]))

[0345] cam=np.maximum(cam,0) / np.max(cam)

[0346] return cam

[0347] #Usage examples

[0348] heatmap=generate_heatmap(model,preprocessed_image,'last_conv_layer_name')

[0349] #Overlay the heat map onto the original image

[0350] def overlay_heatmap(image,heatmap,alpha=0.4):

[0351] heatmap=np.uint8(255*heatmap)

[0352] heatmap=cv2.applyColorMap(heatmap,cv2.COLORMAP_JET)

[0353] superimposed_img=heatmap*alpha+image

[0354] return superimposed_img

[0355] overlay=overlay_heatmap(original_image,heatmap)

[0356] ```

[0357] In this implementation, the embodiment of the present application uses Grad-CAM (Gradient-weighted ClassActivation Mapping) technology to generate a heat map, such as Figure 4 This method can highlight the image area that is most important for model decision making, thereby helping the embodiment of the present application to accurately locate the fault area.

[0358] S4.3: Fault severity assessment

[0359] Based on the fault type, area and duration, the severity of the fault is quantitatively assessed. This is achieved as follows:

[0360] ```Python

[0361] def assess_fault_severity(fault_type,fault_area,duration):

[0362] #Define the basic weight of each fault type

[0363] type_weights={

[0364] 'Pixel failure': 0.5,

[0365] 'Color Abnormal': 0.6,

[0366] 'Brightness unevenness': 0.7,

[0367] 'Flicker': 0.8,

[0368] 'Image Tearing': 0.9,

[0369] 'Module Failure': 1.0

[0370] }

[0371] #Calculate the area factor (assuming the total area is 1)

[0372] area_factor=min(fault_area,1)

[0373] #Calculate duration factor (assuming maximum duration is 24 hours)

[0374] time_factor=min(duration / 24,1)

[0375] # Calculate the overall severity

[0376] severity=type_weights[fault_type]*area_factor*(1+time_factor)

[0377] return min(severity,1) #Limit the severity to between 0 and 1

[0378] #Usage examples

[0379] severity = assess_fault_severity(fault_type,0.1,2) #Assume the fault area is 10% and lasts for 2 hours

[0380] ```

[0381] This evaluation function takes into account the inherent severity of the fault type, the area affected by the fault, and the duration of the fault to give a severity score between 0 and 1.

[0382] S4.4: Dynamically adjust detection sensitivity and threshold

[0383] Based on the fault detection results, the sensitivity and threshold of fault detection are dynamically adjusted. The implementation is as follows:

[0384] ```Python

[0385] class DynamicThresholdAdjuster:

[0386] def __init__(self, initial_threshold = 0.5, adjust_rate = 0.05):

[0387] self.threshold = initial_threshold

[0388] self.adjust_rate = adjust_rate

[0389] self.false_positive_count = 0

[0390] self.false_negative_count = 0

[0391] def adjust_threshold(self, prediction, ground_truth):

[0392] if prediction > self.threshold and ground_truth == 0:

[0393] # False positive

[0394] self.false_positive_count += 1

[0395] self.threshold += self.adjust_rate

[0396] elif prediction <= self.threshold and ground_truth == 1:

[0397] # False negative

[0398] self.false_negative_count += 1

[0399] self.threshold -= self.adjust_rate

[0400] # Ensure the threshold is between 0 and 1

[0401] self.threshold = max(0, min(1, self.threshold))

[0402] def get_threshold(self):

[0403] return self.threshold

[0404] #Usage examples

[0405] adjuster=DynamicThresholdAdjuster()

[0406] #Called after each prediction

[0407] adjuster.adjust_threshold(model_prediction,true_label)

[0408] current_threshold=adjuster.get_threshold()

[0409] ```

[0410] This dynamic threshold adjuster can adaptively adjust the detection threshold based on the model's prediction results and the true label. If there are more false positives, the threshold will be increased; if there are more false negatives, the threshold will be reduced. This can maintain good detection performance in different operating environments and conditions.

[0411] S5: Perform confounding effect analysis to evaluate the impact of environmental factors and operating parameters on the failure.

[0412] Please refer to Figure 3 , which shows a flow chart of the confounding effect analysis provided by an embodiment of the present invention. The analysis includes the following steps:

[0413] S5.1: Detection of variable confounding

[0414] The purpose of variable confounding detection is to analyze the complex relationship between environmental factors (such as temperature, humidity, and light) and operating parameters (such as brightness settings and refresh rates) and the occurrence of failures. This step is crucial to understanding and controlling the root causes of LED display failures. The present application embodiment uses three main methods for variable confounding detection:

[0415] a) Correlation analysis:

[0416] Calculate the Pearson correlation coefficient matrix between all variables. This provides a preliminary understanding of the linear relationship between the variables. For example, the present application embodiment may find that there is a strong positive correlation between ambient temperature and the frequency of faults, while there may be a weak correlation between the display brightness setting and the frequency of faults.

[0417] b) Variance Inflation Factor (VIF) Analysis:

[0418] VIF is used to detect multicollinearity problems. VIF is calculated for each independent variable. If the VIF value is greater than 5 or 10 (depending on the specific criteria), it is considered that the variable may have serious multicollinearity with other variables. For example, the embodiment of the present application may find that the VIF values ​​of ambient temperature and humidity are very high, indicating that these two variables may be strongly correlated and need special attention in subsequent analysis.

[0419] c) Partial correlation analysis:

[0420] Calculate the partial correlation coefficient for each pair of variables, controlling for the effects of other variables. This helps identify direct and indirect relationships. For example, the present application embodiment may find that after controlling for temperature, the correlation between humidity and fault frequency is greatly reduced, suggesting that temperature may be an important mediating variable.

[0421] Through the comprehensive application of these three methods, the embodiments of the present application can fully understand the complex relationship between variables, laying the foundation for subsequent causal inference and quantification of confounding effects.

[0422] Exemplarily, the mixed relationship between environmental factors (such as temperature, humidity, and light) and operating parameters (such as brightness settings and refresh rates) and the occurrence of faults is analyzed. This is achieved as follows:

[0423] ```Python

[0424] import pandas as pd

[0425] import numpy as np

[0426] from statsmodels.stats.outliers_influence import variance_inflation_factor

[0427] def detect_multicollinearity(X):

[0428] vif_data = pd.DataFrame()

[0429] vif_data["Variable"]=X.columns

[0430] vif_data["VIF"]=[variance_inflation_factor(X.values,i)for iin range(X.shape[1])]

[0431] return vif_data

[0432] def partial_correlation(X, y):

[0433] corr_matrix = np.corrcoef(X.T)

[0434] inv_corr_matrix = np.linalg.inv(corr_matrix)

[0435] partial_corr = -inv_corr_matrix / np.sqrt(np.outer(np.diag(inv_corr_matrix), np.diag(inv_corr_matrix)))

[0436] np.fill_diagonal(partial_corr, 1)

[0437] return pd.DataFrame(partial_corr, columns = X.columns, index = X.columns)

[0438] # Usage example

[0439] X = pd.DataFrame({

[0440] 'temperature': temp_data,

[0441] 'humidity': humidity_data,

[0442] 'brightness': brightness_data,

[0443] 'refresh_rate': refresh_rate_data

[0444] })

[0445] y = fault_frequency_data

[0446] vif_results = detect_multicollinearity(X)

[0447] partial_corr_results = partial_correlation(X, y)

[0448] ```

[0449] This implementation includes two main parts: multicollinearity detection (using variance inflation factor VIF) and partial correlation analysis. VIF greater than 5 is usually considered to be significant multicollinearity, and partial correlation analysis can help the embodiment of the application understand the relationship between each pair of variables while controlling other variables.

[0450] S5.2: Causal Inference

[0451] Using causal inference techniques, the present embodiment aims to distinguish and quantify the impact of observable variables and potential unobserved variables on failures. This step is achieved using the structural equation model (SEM):

[0452] a) Model construction:

[0453] Construct a structural equation model containing latent variables. For example, the embodiment of the present application may define two latent variables: "environmental factors" (consisting of temperature, humidity, and light) and "operational factors" (consisting of brightness settings and refresh rates). Then, the embodiment of the present application assumes that these two latent variables jointly affect the fault frequency.

[0454] b) Parameter estimation:

[0455] The model parameters are estimated using the maximum likelihood estimation method. This includes the path coefficients from the latent variables to the observed variables, as well as the relationship coefficients between the latent variables.

[0456] c) Model evaluation:

[0457] The model fit was evaluated, including indicators such as chi-square test, comparative fit index (CFI), and root mean square error approximation (RMSEA). These indicators help the present embodiment to determine whether the model explains the observed data well.

[0458] d) Path analysis:

[0459] Analyze path coefficients and effect sizes to quantify the direct and indirect effects of observed and latent variables on failures. For example, the present application embodiment may find that the total effect of "environmental factors" on failure frequency is greater than that of "operational factors", but "operational factors" have a greater direct effect.

[0460] Through SEM analysis, the embodiments of the present application can reveal a complex causal network and understand the impact mechanism of different factors on LED display screen failures.

[0461] Exemplarily, causal inference techniques are used to distinguish and quantify the impact of observable variables and potential unobserved variables on failures. The present application embodiment uses a structural equation model (SEM) to achieve this goal:

[0462] ```Python

[0463] from semopy import Model

[0464] def causal_inference_sem(data):

[0465] model_spec="""

[0466] # Latent variables

[0467] EnvironmentalFactor=~temperature+humidity+illumination

[0468] OperationalFactor=~brightness+refresh_rate

[0469] #Structural model

[0470] FaultFrequency~EnvironmentalFactor+OperationalFactor

[0471] #Residual correlation (unobserved confounding)

[0472] temperature~~brightness

[0473] """

[0474] model = Model(model_spec)

[0475] model.fit(data)

[0476] results = model.inspect()

[0477] return results

[0478] #Usage examples

[0479] sem_results=causal_inference_sem(collected_data)

[0480] ```

[0481] This SEM model defines two potential variables (environmental factors and operational factors) and assumes that they jointly affect the fault frequency. By analyzing the model parameters and fit, the embodiment of the present application can quantify the impact of observed variables and potential variables on faults.

[0482] S5.3: Assessment of strength of confounding bias

[0483] Calculate the relative strength of confounding bias between different sets of variables and identify key influencing factors. This step is achieved using partial R-squared analysis:

[0484] a) Full model construction:

[0485] A full linear regression model including all predictor variables is constructed and its R-squared value is calculated. This represents the extent to which all variables together explain the variation in failures.

[0486] b) Variable removal:

[0487] Remove each predictor variable one by one, build the reduced models, and calculate the R-squared value for each reduced model.

[0488] c) Partial R-square calculation:

[0489] Calculate the partial R-squared value for each predictor variable, which is the difference between the full model R-squared and the corresponding reduced model R-squared. This represents the unique contribution of each variable.

[0490] d) Contribution comparison:

[0491] Compare the partial R-squared values ​​of different predictor variables to determine the unique contribution of each variable to the frequency of fault occurrence. For example, the embodiment of the present application may find that the partial R-squared value of ambient temperature is the highest, indicating that it is the most critical single factor affecting the frequency of faults.

[0492] Through this analysis, the embodiments of the present application can quantify the unique impact of each factor on LED display screen failure, providing precise guidance for fault prevention and system optimization.

[0493] Exemplarily, the relative strength of confounding bias between different sets of variables is calculated to identify key influencing factors. This embodiment of the application uses partial R-squared analysis to achieve this goal:

[0494] ```Python

[0495] from sklearn.linear_model import LinearRegression

[0496] def partial_r_squared(X,y):

[0497] full_model=LinearRegression().fit(X,y)

[0498] full_r2=full_model.score(X,y)

[0499] partial_r2 = {}

[0500] for i,col in enumerate(X.columns):

[0501] X_reduced=X.drop(col,axis=1)

[0502] reduced_model=LinearRegression().fit(X_reduced,y)

[0503] reduced_r2=reduced_model.score(X_reduced,y)

[0504] partial_r2[col]=full_r2-reduced_r2

[0505] return partial_r2

[0506] #Usage examples

[0507] partial_r2_results=partial_r_squared(X,y)

[0508] ```

[0509] Partial R-squared analysis can help the present embodiment understand the unique contribution of each variable to the frequency of fault occurrence. A higher partial R-squared value indicates that the variable has a stronger explanatory power for the fault frequency.

[0510] S5.4: Optimize operating environment and parameter settings

[0511] Based on the results of the hybrid effect analysis, the embodiment of the present application can optimize the operating environment and parameter settings of the LED display. This step includes:

[0512] a) Environmental control suggestions:

[0513] According to the SEM analysis results, if environmental factors have a significant impact on the failure frequency, the embodiments of the present application may recommend strengthening temperature and humidity control, or adjusting the operating parameters of the display screen under specific environmental conditions.

[0514] b) Operation parameter optimization:

[0515] Based on the partial R-squared analysis results, the embodiments of the present application can determine which operating parameters have the greatest impact on the fault and make optimization suggestions accordingly. For example, if it is found that the brightness setting has a great impact, the embodiments of the present application may suggest reducing the maximum brightness or adjusting the brightness curve under certain conditions.

[0516] c) Interaction effect treatment:

[0517] By testing the results of variable confounding, the embodiments of the present application can identify important interaction effects. For example, if a significant interaction is found between temperature and brightness settings, the embodiments of the present application may suggest automatically reducing the maximum brightness in a high temperature environment.

[0518] d) Preventive maintenance strategy:

[0519] Based on the causal inference results, the embodiments of the present application can formulate more targeted preventive maintenance strategies. For example, if certain factors are identified as early indicators of failure, the embodiments of the present application can establish an early warning system based on these factors.

[0520] e) Adaptive control system:

[0521] Ultimately, the embodiments of the present application can design an adaptive control system that can monitor key environmental and operating parameters in real time and dynamically adjust the operating parameters of the display screen based on the results of the confounding effect analysis to minimize the risk of failure and optimize performance.

[0522] Through this optimization method based on data and causal analysis, the embodiment of the present application can significantly improve the reliability and performance of LED display screens, while reducing maintenance costs and downtime. This method can not only solve current problems, but also provide valuable insights for future product design and system optimization.

[0523] For example, based on the mixed effect analysis results, the operating environment and parameter settings of the LED display are optimized. Based on the previous analysis results, specific optimization suggestions are given. For example:

[0524] ```Python

[0525] def optimize_settings(sem_results,partial_r2_results):

[0526] recommendations=[]

[0527] #Recommendations based on SEM results

[0528] if sem_results['EnvironmentalFactor']['FaultFrequency']>0.5:

[0529] recommendations.append("Consider improving environmental conditions, especially temperature and humidity control")

[0530] if sem_results['OperationalFactor']['FaultFrequency']>0.5:

[0531] recommendations.append("Optimize operating parameters, especially brightness and refresh rate settings")

[0532] #Suggestions based on partial R-squared results

[0533] most_important_factor=max(partial_r2_results,key=partial_r2_results.get)

[0534] recommendations.append(f"Focus on {most_important_factor}, which has the greatest impact on failure frequency")

[0535] return recommendations

[0536] #Usage examples

[0537] optimization_recommendations=optimize_settings(sem_results,partial_r2_results)

[0538] ```

[0539] This function generates a series of optimization suggestions based on the results of SEM analysis and partial R-square analysis. These suggestions can help operation and maintenance personnel adjust the operating environment and parameters of the LED display to reduce the frequency of failures.

[0540] Specifically, taking a certain LED factory as an example, it is necessary to analyze the mixed relationship between variables such as environmental factors and operating parameters and the occurrence of failures.

[0541] Assume that the following variables data are collected:

[0542] X1: Ambient temperature (℃)

[0543] X2: Ambient humidity (%)

[0544] X3: Display brightness setting (%)

[0545] X4: Operating time (hours)

[0546] Y: Failure frequency (times / day)

[0547] The following methods can be used to detect confounding between these variables:

[0548] a) Correlation analysis:

[0549] Calculate the Pearson correlation coefficient matrix between variables. If the absolute value of the correlation coefficient between two independent variables is greater than 0.7, it is considered that they may be confounded.

[0550] b) Variance Inflation Factor (VIF) Analysis:

[0551] Calculate VIF for each independent variable. If VIF>5, it is considered that the variable may be confounded with other variables.

[0552]

[0553] c) Partial correlation analysis:

[0554] Calculate partial correlation coefficients for each pair of variables, controlling for the effects of other variables. This helps identify direct and indirect relationships.

[0555] Separation of observed and unobserved confounders: In this substep, attempts are made to distinguish and quantify the impact of observable variables and potential unobserved variables on the failure.

[0556] For example:

[0557] Structural equation modeling (SEM) was used to separate observed and unobserved confounding effects.

[0558]

[0559]

[0560] Through this model, we can: quantify the direct effects of observed environmental and operational factors on failures, and estimate potential unobserved confounding effects through residual correlations.

[0561] Confounding bias strength assessment: Calculate the relative strength of confounding bias between different sets of variables and identify key influencing factors.

[0562] Specifically, partial R-squared analysis was used to assess the strength of confounding bias.

[0563]

[0564] This analysis can help understand: the unique contribution of each variable to the frequency of failures, and which variables have the strongest confounding effects and need to be focused on.

[0565] Based on the above analysis, the following example conclusions can be drawn:

[0566] 1. There is a strong confounding effect between ambient temperature (X1) and display brightness setting (X3), with a correlation coefficient of 0.75. This may be because in high temperature environments, the system automatically reduces the brightness to protect the hardware.

[0567] 2. The VIF of running time (X4) is 6.2, indicating that it may have multicollinearity with other variables. The present embodiment requires careful interpretation of the effects related to running time.

[0568] 3. The structural equation model shows that the impact of environmental factors on the frequency of failures (standardized coefficient 0.6) is greater than that of operational factors (standardized coefficient 0.4).

[0569] 4. Partial R-square analysis shows that ambient humidity (X2) has the largest unique contribution, explaining 20% ​​of the variation in fault frequency.

[0570] S6: Multi-operator integration optimization

[0571] In this invention, the LSHADE (Linear Success-Historybased Adaptive Differential Evolution) algorithm with restart and local search mechanism is used to optimize model hyperparameters and system configuration. LSHADE is an advanced evolutionary algorithm, which is particularly suitable for solving high-dimensional optimization problems. The following are the key technical details of the LSHADE algorithm:

[0572] 1. Population initialization:

[0573] The initial population is randomly generated within the predefined search space. Each individual represents a possible set of model hyperparameters or system configurations. The population size is usually set to 2-4 times the problem dimension.

[0574] 2. Adaptive parameter control:

[0575] Dynamically adjust the scaling factor F and the crossover probability CR. Use the success rate history to guide parameter tuning. F values ​​are sampled from the Cauchy distribution and CR values ​​are sampled from the normal distribution. The parameter tuning strategy takes into account the historical success rates to balance exploration and exploitation.

[0576] 3. Differential evolution operation:

[0577] For each target vector, three different individuals are randomly selected for differential mutation. The mutation vector is generated by adding the basis vector to the weighted difference vector. The trial vector is generated using a binomial crossover operation.

[0578] 4. Linear population size reduction: As the iterations proceed, the population size is linearly reduced. This helps to focus the search on the most promising areas in the later stages of optimization.

[0579] 5. Archiving mechanism:

[0580] Maintain an external archive to store replaced good solutions. When generating trial vectors, individuals in the archive may be used to increase population diversity.

[0581] 6. Local search mechanism:

[0582] After each iteration, a local search is performed on the current best solution. Local optimization is performed using techniques such as gradient-based methods or pattern search. This helps to fine-tune the good solution that has been found.

[0583] 7. Restart mechanism:

[0584] When the population diversity drops below a predetermined threshold, a restart is triggered. Some of the best individuals are retained and the rest of the population is reinitialized. Restarting helps escape from the local optimum and increases the chance of finding the global optimum.

[0585] 8. Fitness evaluation:

[0586] Build and train a model for each individual (i.e. parameter configuration). Use validation set performance as fitness metric. Consider model complexity as a penalty term to prevent overfitting.

[0587] 9. Select an action:

[0588] An elite selection strategy is adopted. If the trial vector is better than the target vector, the target vector is replaced. This ensures that the population quality improves monotonically over time.

[0589] 10. Termination conditions:

[0590] The predefined maximum number of evaluations is reached. The improvement of the optimal solution is below the threshold for multiple generations. The population diversity decreases below a certain threshold and the number of restarts reaches the upper limit.

[0591] 11. Multi-objective optimization extension:

[0592] Use the Pareto dominance concept to handle multiple optimization objectives. Maintain a set of non-dominated solutions and balance the trade-offs of different objectives.

[0593] 12. Parallel Implementation:

[0594] Use multi-core processors or distributed systems to evaluate individuals in parallel. Use master-slave mode or island mode for parallelization to improve optimization efficiency.

[0595] 13. Constraint processing:

[0596] Use penalty functions or repair strategies to handle parameter constraints. Ensure that the generated solution is always within the valid parameter space.

[0597] 14. Dynamic parameter encoding:

[0598] Different encoding strategies are used for different types of parameters (continuous, discrete, categorical). Gray encoding is used to represent integer parameters to improve the similarity of adjacent values.

[0599] In the LED display fault detection system, the LSHADE algorithm is used to optimize the following aspects:

[0600] 1. Deep learning model hyperparameters:

[0601] Number of convolutional layers, number and size of filters

[0602] Number of units in the GRU layer

[0603] Number of neurons in the fully connected layer

[0604] Learning rate, batch size, activation function selection, etc.

[0605] 2. Feature extraction and preprocessing parameters:

[0606] Parameters of image enhancement algorithms

[0607] Selection and parameters of data augmentation strategies

[0608] 3. Fault detection threshold:

[0609] Detection sensitivity of different types of faults

[0610] Quantification parameters of fault severity

[0611] 4. System operating parameters:

[0612] Display brightness and contrast settings

[0613] Refresh rate and color settings

[0614] Environmental control parameters (such as refrigeration system settings)

[0615] Through this multi-operator integrated optimization method, the embodiment of the present application can efficiently search for the optimal configuration in a complex parameter space, significantly improving the performance and reliability of the LED display screen fault detection system. This method is not only applicable to the initial system configuration, but can also be used for continuous optimization and adaptive adjustment of the system to ensure that the system always maintains the best state under different environments and usage conditions.

[0616] For example, the LSHADE (Linear Success-Historybased Adaptive Differential Evolution) algorithm with restart and local search mechanism is used to optimize model hyperparameters and system configuration. The implementation of the LSHADE algorithm is as follows:

[0617] ```python

[0618] import numpy as np

[0619] from scipy.stats import cauchy

[0620] class LSHADE:

[0621] def__init__(self,func,bounds,pop_size=100,max_evals=10000,p=0.11,H=6):

[0622] self.func=func

[0623] self.bounds=bounds

[0624] self.dim=len(bounds)

[0625] self.pop_size=pop_size

[0626] self.max_evals=max_evals

[0627] self.p=p

[0628] self.H=H

[0629] self.memory_size=self.H*self.dim

[0630] def init_population(self):

[0631] return np.random.uniform(self.bounds[:,0],self.bounds[:,1],size=(self.pop_size,self.dim))

[0632] def mutation(self,x,F):

[0633] return x[0]+F*(x[1]-x[2])

[0634] def crossover(self,x,v,CR):

[0635] mask=np.random.rand(self.dim)<=CR

[0636] y=np.where(mask,v,x)

[0637] return y

[0638] def selection(self,population,offspring):

[0639] return np.where(self.func(offspring)<self.func(population),offspring,population)

[0640] def adapt_parameters(self,memory_f,memory_cr,memory_pos,success_f,success_cr):

[0641] if len(success_f)>0:

[0642] memory_f[memory_pos]=np.mean(success_f)

[0643] memory_cr[memory_pos]=np.mean(success_cr)if np.mean(success_cr)>0else0.5

[0644] memory_pos=(memory_pos+1)%self.memory_size

[0645] return memory_f,memory_cr,memory_pos

[0646] def optimize(self):

[0647] population=self.init_population()

[0648] memory_f=np.ones(self.memory_size)*0.5

[0649] memory_cr=np.ones(self.memory_size)*0.5

[0650] memory_pos=0

[0651] for_in range(self.max_evals / / self.pop_size):

[0652] offspring=np.empty_like(population)

[0653] success_f,success_cr=[],[]

[0654] for iin range(self.pop_size):

[0655] idx_r=np.random.choice(self.memory_size)

[0656] F=cauchy.rvs(loc=memory_f[idx_r],scale=0.1)

[0657] F=np.clip(F,0,1)

[0658] CR=np.random.normal(loc=memory_cr[idx_r],scale=0.1)

[0659] CR=np.clip(CR,0,1)

[0660] x=population[i]

[0661] v=self.mutation(population[np.random.choice(self.pop_size,3,replace=False)],F)

[0662] u=self.crossover(x,v,CR)

[0663] if self.func(u)<self.func(x):

[0664] offspring[i]=u

[0665] success_f.append(F)

[0666] success_cr.append(CR)

[0667] else:

[0668] offspring[i]=x

[0669] population = self.selection(population, offspring)

[0670] memory_f, memory_cr, memory_pos = self.adapt_parameters(memory_f, memory_cr, memory_pos, success_f, success_cr)

[0671] # Local search

[0672] best_idx = np.argmin([self.func(ind) for ind in population])

[0673] population[best_idx] = self.local_search(population[best_idx])

[0674] # Restart mechanism

[0675] if np.std([self.func(ind) for ind in population]) < 1e - 6:

[0676] population = self.init_population()

[0677] best_idx = np.argmin([self.func(ind) for ind in population])

[0678] return population[best_idx], self.func(population[best_idx])

[0679] def local_search(self, x):

[0680] step_size = 0.01

[0681] for i in range(self.dim):

[0682] x_new = x.copy()

[0683] x_new[i] += np.random.uniform(-step_size, step_size)

[0684] x_new[i]=np.clip(x_new[i],self.bounds[i,0],self.bounds[i,1])

[0685] if self.func(x_new) <self.func(x):

[0686] x=x_new

[0687] return x

[0688] #Usage examples

[0689] def objective_function(x):

[0690] return np.sum(x**2)#Here we use a simple spherical function as an example

[0691] bounds = np.array([[-5,5]]*10) #10-dimensional problem, the range of each dimension is [-5,5]

[0692] lshade=LSHADE(objective_function,bounds)

[0693] best_solution,best_fitness=lshade.optimize()

[0694] print("Best solution:",best_solution)

[0695] print("Best fitness:",best_fitness)

[0696] ```

[0697] The LSHADE algorithm implements the following key features:

[0698] 1. Adaptive parameter control: dynamically adjust the scaling factor F and crossover probability CR through the success history.

[0699] 2. Linear population size reduction: Although the population size is not explicitly reduced in this implementation, it can be achieved by modifying the `optimize` method.

[0700] 3. Local search: Perform local search on the current optimal solution to improve the quality of the solution.

[0701] 4. Restart mechanism: When the population diversity drops to a certain level, the population is reinitialized.

[0702] In practical applications, the embodiment of the present application can use the LSHADE algorithm to optimize the hyperparameters of the convolutional gated recurrent deep learning model of the embodiment of the present application, such as the learning rate, convolution kernel size, number of GRU units, etc. The following is a specific application example:

[0703] ```Python

[0704] import tensorflow as tf

[0705] def create_model(params):

[0706] model = tf.keras.Sequential([

[0707] tf.keras.layers.Conv2D(int(params[0]),(int(params[1]),int(params[1])),activation='relu',input_shape=(224,224,3)),

[0708] tf.keras.layers.MaxPooling2D((2,2)),

[0709] tf.keras.layers.Conv2D(int(params[2]),(3,3),activation='relu'),

[0710] tf.keras.layers.MaxPooling2D((2,2)),

[0711] tf.keras.layers.Flatten(),

[0712] tf.keras.layers.Dense(int(params[3]),activation='relu'),

[0713] tf.keras.layers.Dense(6,activation='softmax') ])

[0715] return model

[0716] def objective_function(params):

[0717] model = create_model(params)

[0718] model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=params[4]),

[0719] loss='categorical_crossentropy',

[0720] metrics = ['accuracy'])

[0721] history=model.fit(x_train,y_train,epochs=10,validation_split=0.2,verbose=0)

[0722] return 1-history.history['val_accuracy'][-1]#Minimize the error of the validation set

[0723] #Define parameter range

[0724] bounds = np.array([

[0725] [16,64], #Number of filters in the first convolutional layer

[0726] [3,7],#The kernel size of the first convolutional layer

[0727] [32,128], #The number of filters in the second convolutional layer

[0728] [64,256],#Number of neurons in the fully connected layer

[0729] [1e-4,1e-2]#Learning rate ])

[0731] lshade=LSHADE(objective_function,bounds)

[0732] best_params,best_fitness=lshade.optimize()

[0733] print("Best hyperparameters:",best_params)

[0734] print("Best validation accuracy:",1-best_fitness)

[0735] ```

[0736] In this example, the embodiment of the present application uses the LSHADE algorithm to optimize the hyperparameters of a simple convolutional neural network model. The objective_function creates and trains the model and returns the error rate (1-accuracy) on the validation set. The LSHADE algorithm attempts to minimize this error rate to find the best hyperparameter combination.

[0737] The embodiment of the present invention further provides a LED display screen fault detection system, comprising:

[0738] An acquisition module is used to acquire the display image of the LED display screen;

[0739] A preprocessing module, used for preprocessing the display image;

[0740] A detection module, used for inputting the preprocessed display image into a pre-trained convolutional gated recurrent deep learning model to detect whether the LED display screen has a fault;

[0741] A classification and location module, for classifying and locating the fault if a fault is detected;

[0742] Analysis module for performing confounding effect analysis to evaluate the impact of environmental factors and operating parameters on faults.

[0743] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for detecting faults in an LED display screen, characterized in that: The following steps are involved: Collect the display image of the LED display; Preprocessing the display image; Inputting the preprocessed display image into a pre-trained convolutional gated recurrent deep learning model to detect whether the LED display screen has a fault; If a fault is detected, classifying and locating the fault; Confounding effect analysis was performed to evaluate the impact of environmental factors and operating parameters on the failure.

2. The method according to claim 1, characterized in that The preprocessing of the display image comprises: Performing denoising processing on the collected display image; Performing enhancement processing on the denoised display image; The enhanced display image is subjected to standardization processing.

3. The method according to claim 1, characterized in that The convolutional gated recurrent deep learning model includes: A convolutional layer group, used to extract spatial features of the displayed image; Gated recurrent unit layer to capture temporal features; Fully connected layer, used to integrate spatial and temporal features; The convolution layer group includes multiple convolution blocks, each convolution block includes a 2D convolution layer, a batch normalization layer, a ReLU activation function layer and a maximum pooling layer, and the 2D convolution layer uses convolution kernels of different sizes to extract multi-scale features.

4. The method according to claim 3, characterized in that Before inputting the preprocessed display image into a pre-trained convolutional gated recurrent deep learning model, the method further includes: training the convolutional gated recurrent deep learning model, including: Acquire training data, and perform data enhancement processing on the training data, including random rotation, scaling, brightness and contrast adjustment; The ResNet50 model pre-trained on the ImageNet dataset is used as a feature extractor to extract the initial features of the LED display image; The extracted initial features are input into a convolutional gated recurrent deep learning model to simultaneously optimize the three subtasks of fault detection, classification, and localization.

5. The method according to claim 4, characterized in that The method further comprises: Adding L1 and L2 regularization terms to the loss function to suppress the complexity of the convolutional gated recurrent deep learning model; Use the ReLU6 activation function to limit the activation value to the range of [0,6] to achieve sparse activation; The inception module is used to apply 1x1, 3x3 and 5x5 convolution kernels in parallel to achieve multi-scale feature extraction; The attention mechanism is introduced to dynamically adjust the importance of features of different scales according to the input and realize dynamic weight adjustment.

6. The method according to claim 1, characterized in that The classifying and locating the fault includes: Classifying the detected faults, including at least one or more combinations of pixel failure, color anomaly, uneven brightness, flicker, image tearing, and module failure; Use attention mechanism and heat map technology to accurately locate the location and scope of the fault area; Quantify the severity of the fault based on fault type, area and duration; Based on the fault detection results, the sensitivity and threshold of fault detection are dynamically adjusted.

7. The method according to claim 1, characterized in that The confounding effect analysis includes: Conduct variable confounding detection to analyze the confounding relationship between environmental factors and operating parameters and fault occurrence; Use causal inference techniques to distinguish and quantify the impact of observable and potential unobserved variables on failures; Calculate the relative strength of confounding bias between different sets of variables and identify key influencing factors; Based on the results of the hybrid effect analysis, the operating environment and parameter settings of the LED display are optimized; The variable confounding detection includes calculating the Pearson correlation coefficient matrix between variables, calculating the variance inflation factor for each independent variable, and performing partial correlation analysis to identify direct and indirect relationships.

8. The method according to claim 7, characterized in that The method of distinguishing and quantifying the influence of observable variables and potential unobservable variables on failure is implemented by using structural equation model, including: Construct a structural equation model specification that includes latent variables and define the relationship between observed and latent variables; The structural equation model parameters were estimated using the maximum likelihood estimation method; The goodness of fit of the structural equation model was assessed, including the chi-square test, comparative fit index (CFI), and root mean square error of approximation (RMSEA); Analyze path coefficients and effect sizes to quantify the impact of observed and latent variables on failures; The calculation of the relative strength of confounding bias between different sets of variables is achieved using partial R-square analysis, including: Construct a full linear regression model including all predictor variables; Calculate the R-squared value of the complete linear regression model; Remove each predictor variable one by one and build a reduced model; Calculating the R-squared value for each of the reduced models; Calculate the partial R-squared value for each predictor variable, including the difference between the full model R-squared and the corresponding reduced model R-squared; Compare the partial R-squared values ​​for different predictor variables to determine the unique contribution of each variable to the frequency of failure.

9. The method according to claim 1, characterized in that: The method further comprises: The LSHADE algorithm with restart and local search mechanism is used to optimize model hyperparameters and system configuration. The specific steps include: Initialize the population and algorithm parameters; Evaluate the objective function value for each individual; Applying differential evolution operation to generate test vectors; Adaptively adjust the scaling factor and crossover probability based on the success rate history; Perform local search to improve the current optimal solution; Update population and archive; Check restart conditions and reinitialize the population if necessary; Repeat the above steps until the termination condition is reached.

10. A LED display screen fault detection system, characterized in that: include: An acquisition module is used to acquire the display image of the LED display screen; A preprocessing module, used for preprocessing the display image; A detection module, used for inputting the preprocessed display image into a pre-trained convolutional gated recurrent deep learning model to detect whether the LED display screen has a fault; A classification and location module, for classifying and locating the fault if a fault is detected; Analysis module for performing confounding effect analysis to evaluate the impact of environmental factors and operating parameters on faults.

Images