A method for monitoring the operating status of a glass-based display screen

Abstract

This invention discloses a method for monitoring the operational status of a glass-based display screen, belonging to the field of display screen monitoring technology. The method includes the following steps: S1, collecting various sensor parameters and image parameters during the operation of the glass-based display screen; S2, preprocessing the collected sensor parameters and image parameters; S3, extracting features from the preprocessed data; S4, training an improved BP neural network model based on the extracted feature parameters; and S5, using the improved BP neural network model for status judgment and early warning. This invention employs the above-mentioned method for monitoring the operational status of a glass-based display screen, which can reflect the operational status of the glass-based display screen from multiple dimensions. It overcomes the shortcomings of existing methods that monitor only a single parameter, comprehensively reflects the operational status, and provides timely early warning of potential faults, while also possessing high accuracy.

Description

Technical Field

[0001] This invention relates to the field of display screen monitoring technology, and in particular to a method for monitoring the operating status of a glass-based display screen. Background Technology

[0002] In the current technology, with the continuous development of display technology, glass-based displays are widely used in various fields due to their excellent display effect and stability. However, during long-term use, glass-based displays are prone to various malfunctions, affecting their normal operation. Currently, the methods for monitoring the operating status of glass-based displays are relatively simple, mostly only able to monitor one or a few parameters, making it difficult to comprehensively reflect the operating status of the display and unable to provide timely warnings of potential faults.

[0003] Existing monitoring methods have the following shortcomings: monitoring parameters are one-sided and cannot comprehensively assess the overall operating status of the display screen; data processing methods are simple and have poor filtering effect on interference information; and there is a lack of effective prediction mechanisms, making it difficult to detect potential faults in advance, resulting in failure to deal with faults in a timely manner and causing significant losses. Summary of the Invention

[0004] The purpose of this invention is to provide a method for monitoring the operating status of a glass-based display screen, which aims to solve the problems of existing methods for monitoring the operating status of glass-based display screens, such as limited monitoring parameters, inability to comprehensively reflect the operating status, and inability to provide timely warnings of potential faults.

[0005] To achieve the above objectives, the present invention provides a method for monitoring the operating status of a glass-based display screen, comprising the following steps: S1. Collect various sensor parameters and image parameters during the operation of the glass-based display screen; S2. Perform data preprocessing on the collected sensor parameters and image parameters; S3. Extract features from the preprocessed data; S4. Train the improved BP neural network model based on the extracted feature parameters; S5. Use an improved BP neural network model for state judgment and early warning.

[0006] Preferably, in S1, a main temperature sensor and a humidity sensor are installed at the edge of the glass-based display screen, and a redundant temperature sensor is added to the core heat-generating area; a main brightness sensor is evenly distributed around the backlight module and a redundant brightness sensor is set in the center; a main current sensor and a voltage sensor are installed at the circuit interface, and an additional set of redundant current sensors is deployed at the main circuit interface. The system collects initial data from each sensor, calculates the deviation value of sensors of the same type, and performs consistency calibration on each sensor. It sets an adaptive acquisition trigger mechanism, presets a normal parameter fluctuation threshold, and automatically activates the corresponding redundant sensor for high-frequency acquisition when the data of each main sensor exceeds the threshold. It sets a failure judgment standard, and when the deviation between the data collected by each main sensor and the redundant sensor for n consecutive times is greater than the set failure judgment standard, the main sensor is judged to have failed, the system automatically switches to the redundant sensor as the data source, and records the sensor failure warning information.

[0007] Preferably, in S1, a high-definition industrial camera with a resolution of not less than 1920×1080 is selected as the main camera. The main camera integrates a multispectral acquisition module for acquiring visible light and near-infrared images, and an auxiliary camera is set on one side of the main camera. The main camera is mounted 1-1.5 meters in front of the glass-based display screen, with the lens facing the center of the display screen; the auxiliary camera is deployed at a 45° angle to the upper side to capture edge fault images from the side; image coordinate mapping is achieved through binocular vision calibration to locate the edge fault images from the side view to the main view coordinate system. The multispectral acquisition module employs a weighted fusion algorithm to assign a 1.2-fold weight to areas with abnormal thermal distribution in the acquired near-infrared images in the visible light images, and simultaneously marks them as key monitoring areas to achieve correlation analysis between "display defects and thermal anomalies". Set the base acquisition frequency of the main camera and auxiliary camera to 25 frames per second, and set intelligent exposure control: automatically adjust the exposure time by analyzing the average brightness of the image in real time; when the main current sensor detects an abnormal current or the main temperature sensor detects an abnormal temperature, trigger the image acquisition frame rate to be temporarily increased to 50 frames per second to capture the instantaneous fault.

[0008] Preferably, in step S2, the preprocessing of sensor parameters includes the following steps: Noise reduction: An adaptive Kalman filter algorithm is used to process the acquired sensor data. The Kalman gain is dynamically adjusted by calculating the data fluctuation variance in real time to enhance the filtering ability against sudden interference. At the same time, wavelet threshold denoising is introduced as an auxiliary method to suppress high-frequency noise that still exists after filtering. Outlier repair: Outlier data points are identified using the Isolation Forest algorithm. For single-point outliers, the weighted average of three adjacent valid data points is used for repair. For 2-3 consecutive outliers, redundant sensor data or LSTM predictions based on historical trends are used to fill in the gaps, avoiding data breaks that could affect subsequent analysis. Normalization: The processed sensor data is mapped to the [0, 1] interval using the min-max normalization method, using the formula: ; In the formula, This represents the standardized data after normalization. x Represents raw sensor data, x min This indicates the minimum value of the parameter under normal operating conditions. x max This indicates the maximum value of the parameter under normal operating conditions; A new dynamic range calibration has been added, which updates every 24 hours based on the normal operation data of the day. x min and x max This avoids normalization bias caused by sensor drift during long-term use.

[0009] Preferably, in S2, the image parameter preprocessing includes the following steps: Noise Removal: An adaptive median filtering algorithm is used to dynamically adjust the filtering window according to the noise density while preserving image edge information; for the stripe noise unique to the display screen, Fourier transform filtering is used to locate and suppress the peak frequency corresponding to the stripes in the frequency domain. Image enhancement: A new adaptive contrast enhancement method has been added, which improves details in dark areas through local histogram equalization, while introducing adaptive threshold control to avoid noise amplification caused by over-enhancement; for backlit areas during acquisition, the Retinex algorithm is used for illumination compensation to balance the brightness differences in different areas of the display screen. Image cropping and conversion: By identifying the border position of the display screen, the effective display area is cropped and retained. A distortion correction step is added to geometrically correct the image edge distortion caused by the shooting angle. The cropped color image is converted into a grayscale image while retaining the brightness weight information of the RGB three channels.

[0010] Preferably, in S3, the sensor parameter feature extraction includes: Basic feature extraction: Extract temperature change rate, humidity fluctuation amplitude, brightness uniformity, current stability, and voltage fluctuation coefficient from the preprocessed sensor parameters; where temperature change rate is the change in temperature per unit time; humidity fluctuation amplitude is the maximum difference in humidity within a set time; brightness uniformity is the degree of deviation in brightness of different areas of the display screen; current stability is the standard deviation of the current; and voltage fluctuation coefficient is the coefficient of variation of the voltage. Temporal correlation feature extraction: Set a 5-minute sliding window to calculate the cross-correlation coefficient between temperature and current, and the hysteresis coefficient between humidity and brightness decay; Image parameter feature extraction includes: Basic feature extraction: Image segmentation technology is used to process grayscale images, count the number of bad pixels and the length of missing lines in the image, and calculate the clarity of the displayed content; Spatial topology feature extraction: Construct a grid heatmap based on the distribution of bad points, and extract the clustering degree of bad points and the number of connected components; perform edge fitting on the line defect area and calculate the fractal dimension of the defect contour.

[0011] Preferably, S3 also includes feature importance pre-screening: the ReliefF algorithm is used to calculate the importance score of each feature, a screening threshold is set, and bad spot clustering, current stability and temperature change rate are retained; redundant features are dimensionality reduced to reduce the number of features input to the model and reduce computational complexity.

[0012] Preferably, step S4 specifically includes the following steps: S41. Basic Sample Collection: Collect the characteristic parameters of the glass-based display screen under normal operation and different fault states as training samples, and label the corresponding operating state. S42. Cross-model migration sample processing: Introduce fault data of 3-5 different sizes of displays in the same series, and eliminate model differences through the maximum mean difference MMD feature alignment algorithm; S43. Dataset Integration: Integrate the basic samples and the processed cross-model transfer samples to form a complete training dataset; S44. Model Construction: Construct an improved BP neural network model with the number of input layer nodes equal to the number of filtered feature parameters, two hidden layers, and the number of output layer nodes equal to the number of running state categories; introduce an attention mechanism module and a residual connection structure on the basis of the traditional BP network. S45. Loss function setting: During the training phase, a dynamic loss function is used, introducing weighted cross-entropy for slightly anomalous samples and adding focus loss for severely anomalous samples. S46. Model Training: Train the improved BP neural network model using training samples and optimize the network parameters using gradient descent until the model's prediction accuracy reaches or exceeds the preset threshold. S47. Adversarial verification: Generating simulated fault feature samples through generative adversarial networks, and conducting anti-interference tests on the trained improved BP neural network model to eliminate model parameters that are sensitive to pseudo-features. S48. Model Iteration and Update Mechanism: Set iteration trigger conditions. When the cumulative number of newly added fault samples reaches 500 or the model prediction accuracy is below 90% for 10 consecutive days, incremental training will be started. Full training will be carried out every 6 months to integrate the latest fault data with historical samples to ensure that the improved BP neural network model adapts to the changes in fault modes during long-term operation.

[0013] Preferably, step S5 specifically includes the following steps: S51. Input the filtered feature parameters into the trained improved BP neural network model, and the model outputs the running status on the display screen. S52. Based on the output results, reverse call the feature contribution analysis to locate the core abnormal parameters; add a historical fault case matching mechanism to match the core abnormal parameters with the historical fault database to generate a fault cause probability list with case support. S53. Implement early warning measures.

[0014] Preferably, S53 specifically includes the following steps: S531. Adaptively adjust the warning sensitivity based on the cumulative running time of the display screen and the ambient humidity level; S532. If the abnormality is determined to be minor, a yellow warning signal will be issued through the indicator light built into the display screen, and the temperature adjustment module built into the display screen will be triggered. If the abnormality is determined to be serious, in addition to the red warning signal from the indicator light, a warning message containing "fault location coordinates + emergency handling suggestions" will be sent to the monitoring terminal through the wireless communication module. At the same time, the raw data of the 10 minutes before the abnormality occurred will be automatically recorded for subsequent analysis.

[0015] Therefore, the beneficial effects of the above-mentioned method for monitoring the operating status of a glass-based display screen are as follows: (1) Comprehensive monitoring: It integrates sensor parameters such as temperature, humidity, brightness, current, and voltage as well as display image information, and adds multispectral images and auxiliary viewing angle data, which can reflect the operating status of the glass-based display screen from multiple dimensions, overcoming the shortcomings of existing methods that monitor only one parameter.

[0016] (2) High accuracy: The reliability of data is improved by redundant sensor calibration and dynamic acquisition mechanism, and interference information is removed by preprocessing methods such as adaptive Kalman filtering; the improved BP neural network intelligent model is used for state judgment, and the judgment accuracy is further improved by adding feature screening.

[0017] (3) Strong adaptability: The sensor and image acquisition parameters can be dynamically adjusted according to the actual operating status. The model has the ability to iterate and update, and can adapt to different environments and fault scenarios, which enhances the practicality of the method.

[0018] (4) Excellent operability: It clarifies specific rules such as the failure judgment of redundant sensors and the fusion of multispectral data, and supplements the model iteration mechanism, making the technical solution easier to implement.

[0019] (5) Early warning: It can detect potential faults in advance by changing characteristic parameters and issue different levels of warnings according to the severity of the fault, so as to facilitate timely maintenance measures and reduce the losses caused by the fault.

[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0021] Figure 1 This is a flowchart of an embodiment of a glass-based display screen operation status monitoring method according to the present invention. Detailed Implementation

[0022] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0023] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0024] Example 1 like Figure 1 As shown, the present invention provides a method for monitoring the operating status of a glass-based display screen, comprising the following steps: S1. Collect various sensor parameters and image parameters during the operation of the glass-based display screen.

[0025] Main temperature and humidity sensors are installed at the edge of the glass-based display screen, and redundant temperature sensors are added to the core heat-generating area. Main brightness sensors are evenly distributed around the backlight module, and redundant brightness sensors are set in the center. Main current and voltage sensors are installed at the circuit interface, and an additional set of redundant current sensors is deployed at the main circuit interface.

[0026] In this embodiment, the temperature sensor is a DS18B20 type (accuracy ±0.5℃). One main sensor is deployed at each of the four corners (5cm away from the frame) and the center of the 65-inch glass-based display screen. An additional redundant sensor is deployed in the core heat-generating area (the screen position corresponding to the backlight module driver board). The sensors are fixed by thermally conductive silicone adhesive, and the cables are hidden along the screen frame.

[0027] The humidity sensor is an SHT30 type (accuracy ±2% RH) deployed at the midpoint of the two side bezels of the display screen, 3cm away from the screen surface, to avoid direct contact with the screen and thus avoid measurement deviation.

[0028] The brightness sensor is a BH1750 type (accuracy ±2 lux). Four sensors are evenly distributed in the corresponding screen area of ​​the backlight module (located in the center of the upper left, upper right, lower left, and lower right 1 / 4 area respectively). A redundant sensor is deployed in the center position. The sensor lens is perpendicular to the screen surface and 1 cm away. The current sensor is an ACS712 type current sensor (range 0-5A), and the voltage sensor is an LV25-P type voltage sensor (range 0-30V). They are connected in series in the power supply line of the main circuit interface. A set of redundant current sensors is deployed in parallel next to the main circuit interface, and fast switching is achieved through terminal block connection.

[0029] Initial data from each sensor is collected, deviation values ​​for sensors of the same type are calculated, and consistency calibration is performed on each sensor. An adaptive acquisition trigger mechanism is set, with a preset threshold for normal parameter fluctuations. When the data from any main sensor exceeds the threshold, the corresponding redundant sensor is automatically activated for high-frequency acquisition. A failure judgment criterion is set. When the deviation between the data collected by each main sensor and the redundant sensor for n consecutive times exceeds the set failure judgment criterion, the main sensor is judged to have failed, the system automatically switches to the redundant sensor as the data source, and sensor failure warning information is recorded.

[0030] In this embodiment, the temperature / humidity deviation is calculated as follows: within 5 consecutive acquisition cycles (1 second / time), the deviation rate between the main sensor data T1 and the redundant sensor data T2 is |T1-T2| / T1×100%. When the deviation rate is continuously >10%, a failure judgment is triggered (e.g., if the main sensor displays 25℃ and the redundant sensor displays 28℃, a deviation rate of 12% is sufficient to meet the condition).

[0031] Current / voltage deviation calculation: Similarly, a deviation rate > 5% is considered a failure (e.g., the main sensor displays 2A and the redundant sensor displays 2.12A, a deviation rate of 6% meets the condition). After failure, the system automatically switches to the redundant sensor circuit via a relay and stores a log locally (format: [timestamp] main sensor XX failed, switched to redundant sensor).

[0032] A high-definition industrial camera with a resolution of no less than 1920×1080 is selected as the main camera. The main camera integrates a multispectral acquisition module for acquiring visible light and near-infrared images. An auxiliary camera is set on one side of the main camera.

[0033] In this embodiment, the main camera is a Hikvision MV-CA016-10GM model (1920×1080 resolution), deployed 1.2 meters directly in front of the display screen, with the lens center at the same height as the screen center (1.5 meters from the ground), and fixed with a tripod, with a horizontal deviation ≤±1°. The multispectral module integrates visible light (400-700nm) and near-infrared (800-1000nm) dual-band lenses, with the near-infrared lens focal length matched to the visible light lens to ensure image coordinate alignment in the same area. The multispectral acquisition module uses a weighted fusion algorithm to give 1.2 times the weight to areas with abnormal thermal distribution in the acquired near-infrared image in the visible light image, simultaneously marking them as key monitoring areas to achieve "display defect-thermal anomaly" correlation analysis.

[0034] The auxiliary camera is a Dahua DH-IPC-HFW2120M-I1 model (1280×720 resolution), deployed at a 45° angle above the side of the display screen, 1.5 meters away from the screen body, with the lens facing the connection between the display screen frame and the backlight module, used to collect edge fault images from the side. It is fixed by a bracket and the coordinates are marked (X=1.5m, Y=0.8m, Z=1.8m).

[0035] Image coordinate mapping is achieved through binocular vision calibration, locating the edge fault image from the side view to the main view coordinate system. In this implementation, a checkerboard calibration board (10×10 grid, 2cm on each side) is attached to the surface of the display screen, and the main and auxiliary cameras simultaneously capture 10 sets of calibration images from different angles. The intrinsic parameter matrix (main camera focal length fx=1200 pixels, fy=1200 pixels) and extrinsic parameter matrix (rotation vector and translation vector) are calculated using the calibrateCamera function of the OpenCV library, and a coordinate mapping table (side view pixel coordinates → main view pixel coordinates) is generated, with the calibration error controlled within ±2 pixels. The base acquisition frequency of the main and auxiliary cameras is set to 25 frames per second, and intelligent exposure control is configured: the exposure time is automatically adjusted by analyzing the average image brightness in real time. When the main current sensor detects an abnormal current or the main temperature sensor detects an abnormal temperature, the image acquisition frame rate is temporarily increased to 50 frames per second to capture the momentary fault.

[0036] S2. Perform data preprocessing on the collected sensor parameters and image parameters.

[0037] Sensor parameter preprocessing includes the following steps: Noise reduction: An adaptive Kalman filter algorithm is used to process the acquired sensor data. The Kalman gain is dynamically adjusted by calculating the data fluctuation variance in real time to enhance the filtering capability against sudden interference. In this embodiment, the state transition matrix A=1 (temperature / humidity are slowly varying parameters), the process noise variance Q=0.01, and the initial observation noise variance R=0.1. When the data fluctuation variance (within a 10-second sliding window) > 0.5℃... 2 At that time, R is automatically adjusted to 0.5, and the Kalman gain K is increased synchronously.

[0038] Simultaneously, wavelet threshold denoising is introduced as an auxiliary method to further suppress high-frequency noise that still exists after filtering. The db4 wavelet basis is selected, and the decomposition layer is 3. The high-frequency coefficients are processed with soft thresholding (threshold = 0.6745 × median (|coefficient|)), which effectively suppresses electromagnetic interference from current sensors above 100Hz.

[0039] Outlier repair: Outlier data points are identified using the Isolation Forest algorithm. For single-point outliers, the weighted average of three adjacent valid data points is used for repair. For consecutive outliers of 2-3 points, redundant sensor data or LSTM predictions based on historical trends are used to fill in the gaps, avoiding data breaks that could affect subsequent analysis. In this embodiment, 100 isolation trees are constructed, with a sample subset size of 256. The outlier probability threshold is set to 90%. For single-point outlier repair, the weighted average of the data from the previous 3 seconds (t-3, t-2, t-1) is used (weights 0.5, 0.3, 0.2). For consecutive outliers, the LSTM prediction model is invoked (inputting historical data from the previous 10 minutes and outputting predicted values).

[0040] Normal operation data is collected daily from 0:00 to 0:30 (off-peak hours), and extreme values ​​in the top 1% (e.g., temperature > 35℃) and bottom 1% (e.g., temperature < 15℃) are removed before updating. x min and x max (such as temperature) x min =18℃, x max =32℃), the normalization formula calls the new parameters in real time.

[0041] Normalization: The processed sensor data is mapped to the [0, 1] interval using the min-max normalization method, using the formula: In the formula, This represents the standardized data after normalization. x Represents raw sensor data, x min This indicates the minimum value of the parameter under normal operating conditions. xmax This indicates the maximum value of the parameter under normal operating conditions.

[0042] A new dynamic range calibration has been added, which updates every 24 hours based on the normal operation data of the day. x min and x max This avoids normalization bias caused by sensor drift during long-term use.

[0043] Image parameter preprocessing includes the following steps: Noise Removal: An adaptive median filtering algorithm is employed, dynamically adjusting the filtering window based on noise density while preserving image edge information. For stripe noise specific to the display screen, an additional Fourier transform filter is used to locate and suppress the peak frequencies corresponding to the stripes in the frequency domain.

[0044] In this embodiment, the adaptive median filtering algorithm determines the noise density by statistically analyzing the standard deviation of gray values ​​within a 3×3 window (if the standard deviation is greater than 20, it is determined to be a noisy pixel). If the noise percentage is less than 5%, a 3×3 window is used; otherwise, it is expanded to a 5×5 window. Edge pixels are detected by the Sobel operator (if the gradient value is greater than 30, it is determined to be an edge). A weighted filter of 0.8 × center pixel + 0.2 × neighborhood mean is applied to edge pixels. During Fourier transform filtering, the image is subjected to Fast Fourier Transform (FFT). In the frequency domain, the peak frequency corresponding to the stripe noise is located by power spectrum analysis (usually the peaks in the low-frequency region). The frequency band where the peaks are located is suppressed by a Gaussian mask with a radius of 5. The image is then restored by Inverse Fourier Transform (IFFT).

[0045] Image Enhancement: A new adaptive contrast enhancement method is added, which improves details in dark areas through local histogram equalization. Adaptive threshold control is also introduced to avoid noise amplification caused by over-enhancement. For backlit areas during acquisition, the Retinex algorithm is used for illumination compensation to balance brightness differences in different areas of the display screen. In this embodiment, the image is divided into 16×16 pixel blocks, and histogram equalization is performed on each block, limiting the maximum pixel value increase to ≤30% (e.g., if the original maximum value is 100, it will not exceed 130 after enhancement). For backlit areas (average brightness <50), the Retinex algorithm is used to decompose the image into reflection and illumination components, and gamma correction (γ=0.5) is applied to the illumination component to improve brightness.

[0046] Image cropping and conversion: By identifying the border position of the display screen, the effective display area is cropped and preserved. A distortion correction step is added to geometrically correct image edge distortion caused by the shooting angle. The cropped color image is converted into a grayscale image while retaining the luminance weight information of the RGB three channels.

[0047] S3. Extract features from the preprocessed data.

[0048] Sensor parameter feature extraction includes: Basic Feature Extraction: Temperature change rate, humidity fluctuation amplitude, brightness uniformity, current stability, and voltage fluctuation coefficient are extracted from the preprocessed sensor parameters. Specifically, the temperature change rate is the change in temperature per unit time (calculated as the temperature difference ΔT = Tt - Tt-1 between two consecutive data collection times (t and t-1), where the change rate = ΔT / 1 second (unit: ℃ / s), and the average value is taken over a 10-second sliding window); the humidity fluctuation amplitude is the maximum difference in humidity within a set time period (the difference between the maximum and minimum humidity values ​​Hmax and Hmin within 1 minute (60 data points) = Hmax - Hmin (unit: %RH)); and brightness uniformity is the degree of brightness deviation in different areas of the display screen (the standard deviation σ of the brightness values ​​L1-L4 from the four main sensors is √[(L1-μ)]). 2 +...+(L4-μ) 2 ] / 4, uniformity = σ / μ (μ is the mean value, the smaller the value, the better the uniformity); current stability is the standard deviation of the current; voltage fluctuation coefficient is the coefficient of variation of the voltage.

[0049] Temporal correlation feature extraction: A 5-minute sliding window was set to calculate the cross-correlation coefficient between temperature and current, and the lag coefficient between humidity and brightness decay. The cross-correlation coefficient between temperature and current was calculated using the Pearson formula (taking 300 data points within 5 minutes), and the lag coefficient was obtained by finding the time difference corresponding to the peak value (such as the response delay time of brightness decay after humidity change) through the cross-correlation function.

[0050] Image parameter feature extraction includes: Basic feature extraction: Image segmentation technology is used to process grayscale images, count the number of bad pixels and the length of missing lines in the image, and calculate the clarity of the displayed content.

[0051] In this embodiment, the method for calculating the number of bad pixels is as follows: set a gray value threshold (normal range 50-200), count the total number of pixels with gray values ​​<50 or >200 in the image, and calculate the bad pixel clustering degree by the standard deviation of the bad pixel density in a 3×3 grid (density = number of bad pixels in the grid / 9).

[0052] The method for calculating the length of line defects is as follows: extract the outline of the displayed line through Canny edge detection, locate the straight line using Hough transform, calculate the total break distance of the continuous outline (unit: pixels), and use the box counting method for fractal dimension. The box size ranges from 1×1 to 9×9 pixels. The dimension D = lim (logN) / log(1 / s) (N is the number of boxes covering the outline, and s is the box size).

[0053] Spatial topology feature extraction: Construct a grid heatmap based on the distribution of bad points, and extract the clustering degree of bad points and the number of connected components; perform edge fitting on the line defect area and calculate the fractal dimension of the defect contour.

[0054] Feature importance pre-screening: The ReliefF algorithm is used to calculate the importance score of each feature, and a screening threshold is set to retain bad pixel clustering, current stability, and temperature change rate. Redundant features are dimensionality reduced to decrease the number of features input to the model and reduce computational complexity.

[0055] The ReliefF algorithm parameters are: 100 sampling times, neighborhood radius of 0.1 (after feature value normalization), and weight update formula W=W-(XR) / d+(XH) / d (where X is the target sample, R is the nearest neighbor of the same class, H is the nearest neighbor of different classes, and d is the feature dimension). Screening threshold setting: Calculate the importance score of all features (e.g., bad spot clustering degree 0.8, current stability 0.7, temperature change rate 0.6), retain features with a score >0.3 (usually retain 6-8 core features), and reduce the dimensionality of redundant features with a correlation coefficient >0.8 (e.g., line defect length and fractal dimension) through principal component analysis (PCA), retaining the components with a principal component variance contribution rate >90%.

[0056] S4. Train the improved BP neural network model based on the extracted feature parameters, specifically including the following steps: S41. Basic Sample Collection: Collect the characteristic parameters of the glass-based display screen under normal operation and different fault states as training samples, and label the corresponding operating state.

[0057] In this embodiment, 1000 sets of data were collected (500 sets of normal data, 300 sets of slightly abnormal data, and 200 sets of severely abnormal data). Each set contains 17 original features and corresponding labels (0-normal, 1-slight, 2-severe).

[0058] S42. Cross-model migration sample processing: Introduce 100 sets of fault data for each of 3-5 different sizes (55-inch, 65-inch, 75-inch) displays in the same series, and eliminate model differences by using the maximum mean difference MMD feature alignment algorithm.

[0059] S43. Dataset Integration: Integrate the basic samples and the processed cross-model transfer samples to form a complete training dataset.

[0060] S44. Model Construction: Construct an improved BP neural network model. The number of nodes in the input layer is equal to the number of filtered feature parameters, the hidden layer has 2 layers, and the number of nodes in the output layer is equal to the number of running state categories. Specifically: the number of nodes in the input layer = the number of filtered features (e.g., 8), the first hidden layer has 15 nodes (ReLU activation function), the second hidden layer has 10 nodes (ReLU activation function), and the output layer has 3 nodes (Softmax activation function).

[0061] An attention mechanism module and a residual connection structure are introduced on the basis of the traditional BP network. The attention mechanism assigns initial weights to the input features (bad point clustering degree 0.9, current stability 0.8, temperature change rate 0.7), which are dynamically adjusted through gradient descent during training, and the sum of the weights is normalized to 1.

[0062] S45. Loss Function Settings: A dynamic loss function is used during the training phase. Weighted cross-entropy is introduced for slightly anomalous samples, and focus loss is added for severely anomalous samples. In this embodiment, the weighted cross-entropy weight for slightly anomalous samples (label 1) in the dynamic loss function is 1.5, and the focus loss for severely anomalous samples (label 2) is -α(1-p). 2 log(p) (α=0.75, p is the predicted probability).

[0063] S46. Model Training: Train the improved BP neural network model using training samples, and optimize the network parameters using gradient descent until the model's prediction accuracy reaches or exceeds a preset threshold.

[0064] S47. Adversarial verification: Generating simulated fault feature samples through a generative adversarial network, and conducting anti-interference tests on the trained improved BP neural network model to eliminate model parameters that are sensitive to pseudo-features.

[0065] S48. Model Iteration and Update Mechanism: Set iteration trigger conditions. When the cumulative number of newly added fault samples reaches 500 or the model prediction accuracy is below 90% for 10 consecutive days, incremental training will be started. Full training will be carried out every 6 months to integrate the latest fault data with historical samples to ensure that the improved BP neural network model adapts to the changes in fault modes during long-term operation.

[0066] Incremental training: When 500 new samples are added, the parameters of the first hidden layer are frozen, and only the second and output layers are updated. The learning rate is set to 50% of the initial training (0.005), and the iteration is performed 50 times.

[0067] Full training: Every 6 months, integrate historical samples (retaining 70% of the effective samples) with new samples, reinitialize the parameters of the first layer of the network, with a learning rate of 0.01, iterate 100 times, and save the model with the highest accuracy on the validation set.

[0068] S5. Use an improved BP neural network model for state judgment and early warning, specifically including the following steps: S51. Input the filtered feature parameters into the trained improved BP neural network model, and the model outputs the operating status (normal, slight abnormality, severe abnormality) on the display screen.

[0069] S52. Based on the output results, reverse the feature contribution analysis (calculate the influence weight of each feature on the anomaly judgment through SHAP value) to locate the core anomaly parameters; add a historical fault case matching mechanism to match the core anomaly parameters with the historical fault database (stores fault features, causes and solutions) for similarity matching (using cosine similarity algorithm) to generate a fault cause probability list with case support (such as "backlight module aging (70%, reference case ID: 20230512), poor power interface contact (30%)"), and improve the interpretability of reasoning.

[0070] S53. Implement the early warning, which includes the following steps: S531, Dynamic Threshold Adjustment: Adaptively adjusts the warning sensitivity based on the cumulative running time of the display screen and the ambient humidity level.

[0071] Running time threshold adjustment: After a cumulative running time of 1000 hours, the threshold for slight abnormal brightness uniformity is reduced from 0.15 to 0.14 (reduced by 0.01 each time), and the threshold for humidity fluctuation amplitude is reduced from 5% RH to 4.75%.

[0072] Ambient humidity threshold adjustment: When the humidity sensor detects humidity > 80%, the current stability threshold is tightened from 0.2 to 0.18 (i.e., smaller fluctuations are considered abnormal).

[0073] S532, graded linkage response: If a minor abnormality is detected, a yellow warning signal will be issued through the indicator light built into the display screen, and the temperature regulation module built into the display screen will be triggered (the yellow indicator light flashes once every 2 seconds, triggering the micro cooling fan (12V, speed 3000rpm) on the top of the screen to start and run continuously until the temperature returns to the normal range).

[0074] If a serious anomaly is detected, in addition to the red indicator light warning, a warning message containing "fault location coordinates + emergency handling suggestions" will be sent to the monitoring terminal via the wireless communication module. Specifically, the red indicator light will remain on, and the Wi-Fi module (ESP8266) will send the warning message via the MQTT protocol. The content format is: "Time: 2023-10-01 14:30; Status: Serious Anomaly; Location: Backlight module area (X: 120-240, Y: 80-160 pixels); Cause: Backlight module aging (70%); Suggestion: Check the driver circuit capacitors." At the same time, the raw data of the 10 minutes before the anomaly occurred will be automatically recorded for subsequent analysis.

[0075] Therefore, the above-mentioned method for monitoring the operating status of a glass-based display screen can reflect the operating status of the glass-based display screen from multiple dimensions, overcome the shortcomings of existing methods that only monitor one parameter, comprehensively reflect the operating status and provide timely warnings of potential faults, and has high accuracy.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for monitoring the operating status of a glass-based display screen, characterized in that: Includes the following steps: S1. Collect various sensor parameters and image parameters during the operation of the glass-based display screen; S2. Perform data preprocessing on the collected sensor parameters and image parameters; S3. Extract features from the preprocessed data; S4. Train the improved BP neural network model based on the extracted feature parameters; S5. Use an improved BP neural network model for state judgment and early warning.

2. The method for monitoring the operating status of a glass-based display screen according to claim 1, characterized in that: In S1, a main temperature sensor and a humidity sensor are installed on the edge of the glass-based display screen, and a redundant temperature sensor is added to the core heat-generating area; a main brightness sensor is evenly distributed around the backlight module and a redundant brightness sensor is set in the center; a main current sensor and a voltage sensor are installed at the circuit interface, and an additional group of redundant current sensors is deployed at the main circuit interface. Collect initial data from each sensor, calculate the deviation value of sensors of the same type, and perform consistency calibration on each sensor; An adaptive acquisition triggering mechanism is set up, and a normal parameter fluctuation threshold is preset. When the data of each main sensor exceeds the threshold, the corresponding redundant sensor is automatically activated to perform high-frequency acquisition. A failure judgment criterion is set. When the deviation between the data collected by each main sensor and the redundant sensor for n consecutive times exceeds the set failure judgment criterion, the main sensor is judged to be faulty, the redundant sensor is automatically switched as the data source, and the sensor failure warning information is recorded.

3. The method for monitoring the operating status of a glass-based display screen according to claim 2, characterized in that: In S1, a high-definition industrial camera with a resolution of no less than 1920×1080 is selected as the main camera. The main camera integrates a multispectral acquisition module for acquiring visible light and near-infrared images. An auxiliary camera is set on one side of the main camera. The main camera is mounted 1-1.5 meters in front of the glass-based display screen, with the lens facing the center of the display screen; the auxiliary camera is deployed at a 45° angle to the upper side to capture edge fault images from the side; image coordinate mapping is achieved through binocular vision calibration to locate the edge fault images from the side view to the main view coordinate system. The multispectral acquisition module employs a weighted fusion algorithm to assign a 1.2-fold weight to areas with abnormal thermal distribution in the acquired near-infrared images in the visible light images, and simultaneously marks them as key monitoring areas to achieve correlation analysis between "display defects and thermal anomalies". Set the base acquisition frequency of the main camera and auxiliary camera to 25 frames per second, and set intelligent exposure control: automatically adjust the exposure time by analyzing the average brightness of the image in real time; When the main current sensor detects an abnormal current or the main temperature sensor detects an abnormal temperature, the image acquisition frame rate is temporarily increased to 50 frames per second to capture the instantaneous fault.

4. The method for monitoring the operating status of a glass-based display screen according to claim 1, characterized in that: In S2, the sensor parameter preprocessing includes the following steps: Noise reduction: An adaptive Kalman filter algorithm is used to process the acquired sensor data. The Kalman gain is dynamically adjusted by calculating the data fluctuation variance in real time to enhance the filtering ability against sudden interference. At the same time, wavelet threshold denoising is introduced as an auxiliary method to suppress high-frequency noise that still exists after filtering. Outlier repair: Outlier data points are identified using the Isolation Forest algorithm. For single-point outliers, the weighted average of three adjacent valid data points is used for repair. For 2-3 consecutive outliers, redundant sensor data or LSTM predictions based on historical trends are used to fill in the gaps, avoiding data breaks that could affect subsequent analysis. Normalization: The processed sensor data is mapped to the [0, 1] interval using the min-max normalization method, using the formula: ； In the formula, This represents the standardized data after normalization. x Represents raw sensor data, x min This indicates the minimum value of the parameter under normal operating conditions. x max This indicates the maximum value of the parameter under normal operating conditions; A new dynamic range calibration has been added, which updates every 24 hours based on the normal operation data of the day. x min and x max This avoids normalization bias caused by sensor drift during long-term use.

5. The method for monitoring the operating status of a glass-based display screen according to claim 1, characterized in that: In S2, the image parameter preprocessing includes the following steps: Noise Removal: An adaptive median filtering algorithm is used to dynamically adjust the filtering window according to the noise density while preserving image edge information; for the stripe noise unique to the display screen, Fourier transform filtering is used to locate and suppress the peak frequency corresponding to the stripes in the frequency domain. Image enhancement: A new adaptive contrast enhancement method has been added, which improves details in dark areas through local histogram equalization, while introducing adaptive threshold control to avoid noise amplification caused by over-enhancement; for backlit areas during acquisition, the Retinex algorithm is used for illumination compensation to balance the brightness differences in different areas of the display screen. Image cropping and conversion: By identifying the border position of the display screen, the effective display area is cropped and retained. A distortion correction step is added to geometrically correct the image edge distortion caused by the shooting angle. The cropped color image is converted into a grayscale image while retaining the brightness weight information of the RGB three channels.

6. The method for monitoring the operating status of a glass-based display screen according to claim 1, characterized in that: In S3, sensor parameter feature extraction includes: Basic feature extraction: Extract temperature change rate, humidity fluctuation amplitude, brightness uniformity, current stability, and voltage fluctuation coefficient from the preprocessed sensor parameters; where temperature change rate is the change in temperature per unit time; humidity fluctuation amplitude is the maximum difference in humidity within a set time; brightness uniformity is the degree of deviation in brightness of different areas of the display screen; current stability is the standard deviation of the current; and voltage fluctuation coefficient is the coefficient of variation of the voltage. Temporal correlation feature extraction: Set a 5-minute sliding window to calculate the cross-correlation coefficient between temperature and current, and the hysteresis coefficient between humidity and brightness decay; Image parameter feature extraction includes: Basic feature extraction: Image segmentation technology is used to process grayscale images, count the number of bad pixels and the length of missing lines in the image, and calculate the clarity of the displayed content; Spatial topology feature extraction: Construct a grid heatmap based on the distribution of bad points, and extract the clustering degree of bad points and the number of connected components; perform edge fitting on the line defect area and calculate the fractal dimension of the defect contour.

7. The method for monitoring the operating status of a glass-based display screen according to claim 6, characterized in that: S3 also includes feature importance pre-screening: the ReliefF algorithm is used to calculate the importance score of each feature, a screening threshold is set, and bad spot clustering, current stability and temperature change rate are retained; redundant features are dimensionality reduced to reduce the number of features input to the model and reduce computational complexity.

8. The method for monitoring the operating status of a glass-based display screen according to claim 1, characterized in that: S4 specifically includes the following steps: S41. Basic Sample Collection: Collect the characteristic parameters of the glass-based display screen under normal operation and different fault states as training samples, and label the corresponding operating state. S42. Cross-model migration sample processing: Introduce fault data of 3-5 different sizes of displays in the same series, and eliminate model differences through the maximum mean difference MMD feature alignment algorithm; S43. Dataset Integration: Integrate the basic samples and the processed cross-model transfer samples to form a complete training dataset; S44. Model Construction: Construct an improved BP neural network model with the number of input layer nodes equal to the number of filtered feature parameters, two hidden layers, and the number of output layer nodes equal to the number of running state categories; introduce an attention mechanism module and a residual connection structure on the basis of the traditional BP network. S45. Loss function setting: During the training phase, a dynamic loss function is used, introducing weighted cross-entropy for slightly anomalous samples and adding focus loss for severely anomalous samples. S46. Model Training: Train the improved BP neural network model using training samples and optimize the network parameters using gradient descent until the model's prediction accuracy reaches or exceeds the preset threshold. S47. Adversarial verification: Generating simulated fault feature samples through generative adversarial networks, and conducting anti-interference tests on the trained improved BP neural network model to eliminate model parameters that are sensitive to pseudo-features. S48. Model Iteration and Update Mechanism: Set iteration trigger conditions. When the cumulative number of newly added fault samples reaches 500 or the model prediction accuracy is below 90% for 10 consecutive days, incremental training will be started. Full training will be carried out every 6 months to integrate the latest fault data with historical samples to ensure that the improved BP neural network model adapts to the changes in fault modes during long-term operation.

9. The method for monitoring the operating status of a glass-based display screen according to claim 1, characterized in that: S5 specifically includes the following steps: S51. Input the filtered feature parameters into the trained improved BP neural network model, and the model outputs the running status on the display screen. S52. Based on the output results, reverse call the feature contribution analysis to locate the core abnormal parameters; add a historical fault case matching mechanism to match the core abnormal parameters with the historical fault database to generate a fault cause probability list with case support. S53. Implement early warning measures.

10. The method for monitoring the operating status of a glass-based display screen according to claim 9, characterized in that: S53 specifically includes the following steps: S531. Adaptively adjust the warning sensitivity based on the cumulative running time of the display screen and the ambient humidity level; S532. If the abnormality is determined to be minor, a yellow warning signal will be issued through the indicator light built into the display screen, and the temperature adjustment module built into the display screen will be triggered. If the abnormality is determined to be serious, in addition to the red warning signal from the indicator light, a warning message containing "fault location coordinates + emergency handling suggestions" will be sent to the monitoring terminal through the wireless communication module. At the same time, the raw data of the 10 minutes before the abnormality occurred will be automatically recorded for subsequent analysis.

Images