Remote controller liquid crystal screen image detection method based on computer vision

Abstract

The invention relates to the field of computer vision and image processing, and discloses a remote controller liquid crystal screen image detection method based on computer vision, which comprises the following steps: S1, acquiring an original image containing a remote controller, and carrying out gray level conversion and filtering denoising processing on the original image to obtain a preprocessed image; s2, calculating the local information entropy and the change rate of the image through a sliding window under multiple scales based on the preprocessed image, generating a multi-scale entropy gradient heat map, and screening out a candidate region with high entropy difference as a liquid crystal display region according to a set threshold value; and S3, carrying out vectorization partitioning on the candidate region, and constructing an image matrix. According to the invention, by introducing an image preprocessing mode of low-rank sparse decomposition and normalization processing, the expression ability of structural information in the character image is effectively enhanced, and the feature extraction accuracy under the conditions of complex background, low character contrast and the like is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a remote control liquid crystal screen image detection method based on computer vision. Background Art

[0002] In typical application scenarios such as character recognition on remote control LCD screens, image analysis of industrial equipment interfaces, and low-resolution character extraction, the characters embedded in images often exhibit visual characteristics such as small size, simple structure, weak contrast, and strong background interference. The processing of such images faces multiple technical challenges, especially when the character boundaries are blurred, partially occluded, or the image has lighting changes. Traditional image segmentation and template matching methods are difficult to ensure robustness and accuracy.

[0003] Among existing character recognition technologies, some methods rely on regular templates or static morphological matching mechanisms, which can achieve good results in scenarios with clear images and standardized character structures. However, in cases of complex backgrounds or obvious character deformation, such methods often fail to recognize due to the accumulation of matching errors. At the same time, traditional grayscale histogram, edge operator or contour recognition methods have low tolerance for image noise and lack a deep understanding of the local structural features of characters, which limits their practicality and system adaptability.

[0004] In recent years, although neural network-based character recognition methods have gradually emerged, most models rely on large-scale standard training data and lack specialized optimization for low-quality embedded character images. In addition, existing solutions generally ignore the dynamic stability judgment mechanism of recognition results. When processing image frame sequences, the recognition system is easily affected by single-frame noise or image interference, which can cause recognition fluctuations, false triggering of logic control, or misjudgment of system status.

[0005] In summary, when facing embedded display characters such as remote control LCD screens, existing character image recognition technologies still have problems such as poor preprocessing robustness, insufficient feature extraction dimensions, unclear recognition credibility, and lack of stability control mechanism. There is an urgent need to provide a new recognition technology solution suitable for weakly structured character scenarios to achieve the goal of high stability and high adaptability of character image recognition and processing in complex environments. Summary of the Invention

[0006] The purpose of the present invention is to provide a remote control LCD screen image detection method based on computer vision, which solves the problems of poor recognition robustness, low feature extraction accuracy and lack of recognition result credibility and stability judgment in existing character image recognition technology when dealing with scenes with weak structures, complex backgrounds, unstable image quality, etc.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a remote control LCD screen image detection method based on computer vision, comprising the following steps: S1. Acquire an original image containing the remote control, and perform grayscale conversion and filtering denoising on the original image to obtain a preprocessed image; S2. Based on the preprocessed image, calculate the local information entropy and its change rate of the image by sliding windows at multiple scales, generate a multi-scale entropy gradient heat map, and screen out candidate areas with high entropy differences as LCD display areas based on a set threshold; S3. Vectorize and divide the candidate region into blocks and construct an image matrix. Apply a low-rank and sparse joint decomposition model including a regularization parameter to the matrix to separate the foreground character structure and background interference, thereby obtaining a sparse reconstructed image after character enhancement. S4. Extracting multidimensional features including directional gradient, structural entropy, and grayscale projection from the candidate character region in the sparsely reconstructed image, and inputting the features into a neural network classification model to obtain a character recognition result and its corresponding confidence score; S5. Based on the confidence score and the structural entropy value of the sparsely reconstructed image, a joint scoring function is constructed and a recognition quality score is calculated. The recognition quality score is compared with a preset scoring threshold. Based on the comparison result, the information entropy calculation scale and the sparse modeling regularization parameter are updated and adjusted to achieve adaptive optimization of image region extraction and reconstruction recognition performance.

[0008] Preferably, in step S1, the grayscale conversion includes converting the original image into a grayscale image, and applying Gaussian filtering to the grayscale image to remove noise.

[0009] Preferably, in step S2, the step of calculating the local information entropy and its change rate of the image by sliding windows at multiple scales and generating a multi-scale entropy gradient heat map includes: Under the preset scale set, a sliding window of corresponding size is used to traverse all pixel areas on the preprocessed image based on each scale, and the grayscale histogram probability distribution of the local image is calculated in each window, and the local information entropy value is calculated accordingly, where the calculation formula of the local information entropy value is: Where L is the grayscale level; (x, y) is the center coordinate of the sliding window; is the local information entropy value; p j is the grayscale histogram probability distribution of the local image; s i For every scale; After obtaining the local information entropy value, the gradient change rate of the entropy value in the spatial position is further calculated at the same scale. The calculation formula is: in, is the scale s i The local information entropy gradient modulus at the coordinate point (x, y); is the gradient of information entropy in the horizontal direction; is the gradient of information entropy in the vertical direction; The gradient maps at each scale are fused according to the preset weights to obtain the final multi-scale entropy gradient heat map. The fusion formula is: Among them, α i is the preset weight and satisfies the normalization condition; T(x,y) is the pixel response value at the coordinate point (x,y) in the final generated multi-scale entropy gradient heat map; n is the number of scales in the scale set; The multi-scale entropy gradient heat map reflects the comprehensive intensity of structural changes of the image at different scales and is used for subsequent candidate region extraction.

[0010] Preferably, in step S2, the step of screening out candidate areas with high entropy differences as LCD display areas according to a set threshold value includes: Based on the generated multi-scale entropy gradient heat map, the response values ​​of all pixel positions in the image are normalized to obtain a normalized response map; The normalized response map is compared with a preset threshold; Based on the comparison results, the connected areas formed by all the pixel coordinate points that meet the requirements are screened out and used as the candidate area set of the LCD display area for subsequent image modeling and recognition processing.

[0011] Preferably, in step S3, the step of vectorizing and dividing the candidate area into blocks and constructing an image matrix includes: Each candidate region is divided into non-overlapping image sub-blocks of size w×h, and each sub-block is concatenated pixel by pixel in row-first order to generate a vector of dimension 1×(w·h), where the generation formula is: v k =vec(B k ); Among them, B k is the k-th image sub-block, whose dimension is w×h; vec(.) is the vectorized operation in row-priority order; v k is the pixel vector of the kth sub-block; All obtained sub-block vectors are concatenated column by column to form an image observation matrix, which is used for subsequent low-rank and sparse joint decomposition processing to enhance the foreground character structure.

[0012] Preferably, in step S3, the step of applying a low-rank and sparse joint decomposition model including a regularization parameter to the matrix to separate the foreground character structure and background interference, thereby obtaining a sparse reconstructed image after character enhancement includes: First, the image observation matrix is ​​decomposed through a low-rank and sparse joint decomposition model to obtain the foreground character structure matrix and the background interference matrix; In the process of image observation matrix decomposition, an objective function containing regularization parameters is applied, and the objective function is: Where D is the image observation matrix; L is a low-rank matrix representing the foreground character structure; S is a sparse matrix representing the background interference part; ‖·‖ F is the Frobenius norm of the matrix, which is used to measure the difference between matrices; ‖L‖ * is the nuclear norm of the low-rank matrix L, which promotes low-rank features; ‖S‖1 is the l1 norm of the sparse matrix S, which promotes sparsity; Through the above decomposition, the foreground character structure matrix and background interference matrix are optimized.

[0013] Preferably, in step S4, the step of extracting multidimensional features including directional gradient, structural entropy and grayscale projection includes: Based on the image restored by the foreground low-rank matrix obtained by the low-rank and sparse joint decomposition model, the following multidimensional feature extraction operations are performed: Extract directional gradient features: The Sobel operator is used to calculate the horizontal gradient and vertical gradient of each pixel in the image, and the gradient magnitude map and gradient direction map are constructed accordingly. Extract structural entropy features: A fixed-size local area is selected with the pixel point as the center, the grayscale probability distribution in the area is statistically analyzed, and the local structural entropy response value of the point is calculated; Extract grayscale projection features: The image is projected with grayscale accumulation along the horizontal direction and the vertical direction respectively to obtain a horizontal projection vector and a vertical projection vector; the directional gradient feature, the structural entropy feature and the grayscale projection feature are obtained to jointly constitute a multi-dimensional feature representation of the image for use in subsequent character positioning or recognition processing.

[0014] Preferably, in step S4, the step of inputting the features into a neural network classification model to obtain a character recognition result and its corresponding confidence score includes: The directional gradient features, structural entropy features and grayscale projection features extracted from the image are spliced ​​in a unified order to construct a one-dimensional feature vector as the input of the neural network classification model; The one-dimensional feature vector is input into the pre-trained neural network classification model, and the predicted probability distribution vector of each character category is obtained through the forward propagation process; Based on the predicted probability distribution vector, the character category corresponding to the maximum probability is used as the character recognition result, and the maximum probability value of the character category is used as the confidence score of the recognition result for subsequent screening and decision-making.

[0015] Preferably, in step S5, the steps of constructing a joint scoring function and calculating the recognition quality score include: Based on the confidence score of the character recognition results and the multi-dimensional structural information of the image region, a joint scoring function is constructed to comprehensively evaluate the recognition quality. The specific steps include: Define the joint scoring function, the function form is: in, is the confidence score of the neural network recognition result; S is the structural stability score of the image region; α and β are the weight coefficients of the joint scoring function; The calculation of the structural stability score S is based on the comprehensive representation of the structural entropy mean and gradient amplitude variance of the image. The calculation rule is: Among them, H(x,y) is the local structural entropy value of the pixel (x,y) in image I; Var(G mag ) is the image gradient magnitude map G mag The global variance of Ω is the pixel set of the character area of ​​the image; N = |Ω| is the number of pixels in the character area; γ∈[0,1] is the fusion coefficient of structural entropy and gradient variance; The joint scoring function represents the overall reliability of the current character recognition result and is used for subsequent screening strategies, result optimization and dynamic threshold control.

[0016] Preferably, in step S5, the step of comparing the recognition quality score with a preset score threshold and updating and adjusting the information entropy calculation scale and the sparse modeling regularization parameter based on the comparison result includes: Set a scoring threshold, compare the recognition quality score with the threshold, and judge the reliability of the current recognition result. The judgment conditions are: Q≥T Q Reliable identification, Q <T Q Identification is unreliable; If the recognition is unreliable, update and adjust the following two key parameters: Information entropy calculation scale parameters: Set the minimum scale, maximum scale and current scale, and use the adaptive step size strategy to increase the scale to enhance local statistical stability. The update rule is: s t+1 =min(s t +δ s ,s max ); Among them, δ s The increment step of the preset scale; s t+1 Calculate the scale of the updated structural entropy; s t is the current scale; s max is the maximum scale; Sparse modeling regularization parameter: Based on the current score gap, the regularization term coefficient is adjusted to improve the robustness of foreground extraction. The update rule is defined as: t+1 =λ t ·(1+η·(1-Q)); Among them, λ t is the current regularization parameter; η is the regularization adjustment rate factor; Q is the joint score value; λ t+1 is the updated regularization parameter; If the recognition is reliable, the current parameters are kept unchanged and the subsequent recognition process continues.

[0017] In summary, the present invention includes at least one of the following beneficial technical effects: 1. The present invention effectively enhances the expressiveness of structural information in character images by introducing an image preprocessing method of low-rank sparse decomposition and normalization, improves the accuracy of feature extraction in cases of complex backgrounds and low character contrast, and provides a more stable image foundation for subsequent recognition modules.

[0018] 2. The present invention constructs an image structure entropy calculation model to quantitatively analyze the grayscale distribution complexity of local image areas, enabling the system to identify and adapt to the texture change characteristics of different image areas, providing theoretical support and calculation basis for regional screening and dynamic adjustment during the recognition process.

[0019] 3. The present invention adopts a classification and recognition architecture based on a neural network and introduces a Softmax scoring mechanism to achieve unified discrimination of character categories and quantitative output of confidence metrics, providing the system with a clear and reliable scoring standard, which facilitates subsequent processing modules to perform adaptive strategy control and error avoidance based on this.

[0020] 4. The present invention constructs a stability judgment and state switching control process by designing a confidence threshold judgment and multi-frame consistency verification mechanism, thereby improving the robustness of the recognition results in continuous image sequences and avoiding false triggering of the control logic caused by instantaneous interference or recognition fluctuations.

[0021] 5. The character image recognition process provided by the present invention has good versatility and adaptability, and can be flexibly applied to a variety of embedded systems or low-power image processing platforms, supporting rapid integration and deployment in scenarios such as actual remote control screen recognition and smart terminal character extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A diagram showing the steps of the method of the present invention. DETAILED DESCRIPTION

[0023] The following is combined with Figure 1 , the present invention is described in further detail.

[0024] like Figure 1 As shown, an embodiment of the present invention provides a remote control LCD screen image detection method based on computer vision. For the sake of clarity and to facilitate subsequent detailed description with reference to the accompanying drawings, the method may include the following steps: S1. Acquire an original image containing the remote control, and perform grayscale conversion and filtering denoising on the original image to obtain a preprocessed image; S2. Based on the preprocessed image, calculate the local information entropy and its change rate of the image by sliding windows at multiple scales, generate a multi-scale entropy gradient heat map, and screen out candidate areas with high entropy differences as LCD display areas based on a set threshold; S3. Vectorize and divide the candidate region into blocks and construct an image matrix. Apply a low-rank and sparse joint decomposition model including a regularization parameter to the matrix to separate the foreground character structure and background interference, thereby obtaining a sparse reconstructed image after character enhancement. S4. Extracting multidimensional features including directional gradient, structural entropy, and grayscale projection from the candidate character region in the sparsely reconstructed image, and inputting the features into a neural network classification model to obtain a character recognition result and its corresponding confidence score; S5. Based on the confidence score and the structural entropy value of the sparsely reconstructed image, a joint scoring function is constructed and a recognition quality score is calculated. The recognition quality score is compared with a preset scoring threshold. Based on the comparison result, the information entropy calculation scale and the sparse modeling regularization parameter are updated and adjusted to achieve adaptive optimization of image region extraction and reconstruction recognition performance.

[0025] Regarding step S1, in this embodiment, image acquisition and preprocessing are fundamental to the entire character detection process. Their quality directly impacts the stability and robustness of subsequent image decomposition, feature extraction, and recognition evaluation. To ensure image data availability and processing consistency, this step includes acquiring the image measurement matrix, performing unified grayscale processing, performing noise suppression filtering, and normalizing the input format.

[0026] In terms of image acquisition, this embodiment preferably uses an image acquisition module or a camera device to shoot the remote control LCD screen in real time to obtain an image observation matrix containing the character area, which is set as Where m and n represent the number of rows and columns of the image respectively.

[0027] The measurement matrix D is the starting point of the subsequent image processing process and is used for low-rank sparse structure modeling and image enhancement processing.

[0028] To improve the image feature extraction effect of subsequent models, the original image data is uniformly grayscaled in this embodiment. The grayscale process can be implemented based on the maximum method, average method, or weighted RGB component transformation. Preferably, the following linear transformation formula is used to convert the color image to the grayscale image: I gray (x,y)=0.299·R(x,y)+0.587·G(x,y)+0.114·B(x,y); Among them, R(x,y), G(x,y) and B(x,y) are the red, green and blue channel values ​​at the pixel point (x,y) respectively.

[0029] Through this weighted grayscale conversion, the data channel dimension can be compressed while retaining the image brightness structure, which facilitates subsequent low-rank sparse modeling.

[0030] This embodiment incorporates a noise suppression mechanism to improve image structural clarity, addressing interference introduced by factors such as ambient light interference and sensor noise during the acquisition process. Preferably, a median filter or a Gaussian filter can be used to smooth grayscale images. Median filtering can remove isolated noise points while preserving edge information, while Gaussian filtering provides excellent control over image background smoothness, making it suitable for preprocessing under uneven lighting conditions.

[0031] During the image preprocessing stage, this embodiment further normalizes the image size to ensure that the input image meets a uniform size specification and reduce the model's sensitivity to image size changes. Preferably, the image matrix D is resampled to a preset uniform resolution of m0×n0 to ensure that the subsequent structural entropy response window and gradient calculation window maintain consistent scale with the regularized modeling structure. Normalization can be achieved using linear interpolation, bilinear interpolation, or regional resampling.

[0032] After completing the above steps, we obtain a uniformly preprocessed image grayscale matrix I, which serves as the input for the subsequent low-rank sparse modeling. This image matrix I is highly consistent and manageable, with relatively low background noise interference and clear structural edges, making it suitable for foreground structure extraction based on matrix decomposition.

[0033] It's worth noting that the initial image input and preprocessing are not only a process of information cleaning, but also provide stable and regular data support for the subsequent feature extraction module. These operations ensure that the image content and distribution numerically meet the basic requirements for subsequent mathematical modeling, especially providing direct support for the subsequent low-rank and sparse joint modeling scheme.

[0034] Furthermore, in this embodiment, no restrictions are placed on hardware type, pixel depth, or sampling frame rate during image acquisition and preprocessing, maintaining the method's versatility and adaptability. The core technical advantage of this invention lies in its algorithmic-level data representation structure and processing mechanism, enabling flexible configuration of acquisition hardware based on application requirements.

[0035] In step S2, in this embodiment, after image acquisition and preprocessing, a structurally uniform, clearly grayscaled image matrix I is obtained as the input data for this step. To effectively separate foreground characters from background noise in the image and improve target distinguishability in subsequent recognition tasks, this step employs a low-rank and sparse joint modeling approach to structurally decouple image content and extract saliency.

[0036] Specifically, in this embodiment, the image matrix I is regarded as the observation matrix D, which is essentially a linear superposition of potential low-rank structures and sparse interference, that is, it satisfies the following model expression: D=L+S; Among them, L is the low-rank structural component in the image, which usually corresponds to the main form and continuous edges of the character area; S represents the sparse component, which includes local background noise, sudden texture and abnormal lighting areas in the image.

[0037] In order to recover the low-rank term L from the measurement matrix D, the sparse robust principal component analysis (RobustPCA) model is preferably used in this embodiment for solution. The specific optimization objective function is as follows: min L,S ‖L‖ * +λ‖S‖1subject to D=L+S; Among them, ‖L‖ * represents the nuclear norm of the matrix L, that is, the sum of all singular values, which is used to encourage low rank; ‖S‖1 represents the l1-norm of the elements of the matrix S, that is, the sum of the absolute values ​​of all elements, which is used to characterize sparsity; λ is the regularization coefficient, which is used to balance the optimization weight between low rank terms and sparse terms.

[0038] The above optimization problem is a typical convex optimization problem with a global optimal solution. To efficiently solve this model, this embodiment can use the alternating direction multiplier method (ADMM) or the augmented Lagrangian multiplier method (ALM), updating L and S separately in each iteration until the convergence condition is met.

[0039] In actual operation, in order to improve the modeling ability to express character structure, a background noise prediction strategy or a support graph constraint mechanism can be introduced in the optimization process to eliminate some repeated textures or edge interference items, so that the final decomposed L has more foreground feature significance and geometric stability.

[0040] The matrix L obtained after the optimization is completed is the extracted foreground character structure image, which retains the main visual components of the character such as edge, shape, stroke outline, etc. in the grayscale domain, and has good structural coherence and contrast.

[0041] It's important to note that the low-rank sparse decomposition model used in this step is not only suitable for images with low contrast between characters and backgrounds, but also exhibits a certain degree of anti-interference capability. Even in complex scenarios with irregular edges, light reflections, or transparent occlusions within the character region, the model can still stably recover the low-rank component L with consistent physical structure, providing a clear foundation for subsequent image structure modeling and recognition scoring.

[0042] In addition, this embodiment does not limit the parameters such as image resolution, singular value cutoff threshold, and maximum number of iterations. The relevant parameters can be dynamically set based on the actual image quality and computing resources. The model itself has certain adaptability and generalization.

[0043] In summary, this embodiment achieves effective structural extraction of foreground characters in the remote control LCD screen image by constructing and solving a low-rank and sparse joint decomposition model, providing a robust input basis for subsequent image feature analysis and recognition confidence scoring.

[0044] Regarding step S3, in this embodiment, in order to achieve accurate recognition and subsequent evaluation of the foreground character structure, it is necessary to perform a multi-dimensional image feature extraction operation on the foreground structure image obtained by low-rank sparse decomposition extraction.

[0045] The image feature extraction step uses the low-rank matrix L obtained in step S2 as input. This matrix primarily reflects the structural outlines and significant edge components of the character region in the grayscale domain. To ensure sufficient discriminative power for the subsequent neural network classification model, this step comprehensively constructs directional gradient features, structural entropy features, and grayscale projection features to form a complete multidimensional description of the image.

[0046] First, normalize the low-rank matrix L to the image form I. The transformation rule is: Among them, I(x,y) is the normalized image pixel value, which represents the standardized image grayscale at the coordinate (x,y); L(x,y) is the pixel value in the original image matrix, which represents the grayscale value at the position (x,y); min(L) is the minimum value of all pixel values ​​in the matrix L; max(L) is the maximum value of all pixel values ​​in the matrix L; L refers to the image matrix obtained after low-rank sparse decomposition, which is a two-dimensional real-valued matrix expressing the character structure in the grayscale domain, with a dimension of m×n, where m and n represent the height and width of the image, respectively.

[0047] After the transformation, the pixel values ​​are normalized to the interval [0, 1], which facilitates subsequent image operations and stable extraction of statistics.

[0048] In this embodiment, the following three types of features are extracted for image I: Directional gradient features: In order to capture the direction information of the character edge and the local shape change, the Sobel gradient operator is preferably used to process the image in this embodiment to calculate the local gradient response in the horizontal and vertical directions respectively. The formula is as follows: G x (x,y)=I(x+1,y)-I(x-1,y); G y (x,y)=I(x,y+1)-I(x,y-1); Among them, G x (x, y) is the image gradient in the horizontal direction, reflecting the degree of grayscale change at the pixel (x, y) along the horizontal direction; G y (x, y) is the image gradient in the vertical direction, reflecting the degree of grayscale change at the pixel (x, y) along the vertical direction; I(x,y) is the grayscale value at position (x,y) in the normalized image; x is the horizontal coordinate of the image; y is the vertical coordinate of the image; I(x+1,y) and I(x-1,y) are the grayscale values ​​of the image at adjacent pixels in the horizontal direction; I(x,y+1) and I(x,y-1) are the grayscale values ​​of the image at adjacent pixels in the vertical direction.

[0049] Based on the first-order derivative information obtained above, the gradient amplitude and gradient direction angle of the pixel point are calculated: Among them, ∈ is a small constant set to prevent the denominator from being zero; G mag (x, y) is the gradient amplitude of the pixel point (x, y); G θ(x,y) is the pixel (x,y) of the pixel (x,y).

[0050] This part of the features reflects the edge strength and contour direction of the image, and can be used to depict the stroke structure and writing trend of the characters.

[0051] Structural entropy characteristics: In order to reflect the information complexity and structural uncertainty of a local area in an image, this embodiment introduces a structural entropy response as an indicator to describe the disorder of image texture.

[0052] A sliding window of size s×s is constructed around each pixel point (x, y) (where s is the set local window size), and the frequency distribution p of each gray level j in the window is counted. j ,According to this, the entropy value response is calculated, and the calculation formula is: Where L is the number of gray levels; δ is a decimal term to prevent log(0).

[0053] The structural entropy response map H(x,y) can be used to analyze the complex regional distribution of the image. The entropy value near the edge of the character is usually higher, thus having a discriminative effect.

[0054] Grayscale projection features: To further obtain the global morphological information of the character distribution in the image, the grayscale projection feature is introduced in this embodiment. The pixel values ​​of the image are summed in the horizontal and vertical directions respectively to construct horizontal and vertical projections. The construction rules are as follows: Among them, P hor (x) is the horizontal grayscale projection value of the image at the position of the horizontal coordinate x; n is the height of the image, that is, the maximum number of pixels in the vertical direction; y is the vertical coordinate index of the image, which is used to traverse all rows in the current column in the projection calculation.

[0055] Among them, P ver (y) is the vertical grayscale projection value of the image at the position with the vertical coordinate y; m is the width of the image, that is, the maximum number of pixels (columns) in the horizontal direction; x is the horizontal coordinate index of the image, which is used to traverse all columns in the current row in the projection calculation.

[0056] The above projection features reflect the main spatial distribution range and energy concentration characteristics of the characters in the image, and can be used to assist in distinguishing the relative positions and spacing patterns between characters.

[0057] After completing the above three types of feature extraction, in this embodiment, all features are dimensionally regularized and vectorized to form a unified one-dimensional feature representation vector. Let the directional gradient feature be g, the structural entropy feature be h, and the grayscale projection feature be p, then the final input vector is represented as: f=[g,h,p]; Among them, f is the combined image feature vector, which can be directly input into the neural network recognition model as the basis for classification.

[0058] It is worth noting that this multi-dimensional image feature extraction scheme not only integrates local details and global structures, but also has certain anti-rotation and anti-scale capabilities. It is suitable for typical application scenarios where characters on the remote control LCD screen image are of different sizes, changing positions, and have complex backgrounds.

[0059] At the same time, this embodiment does not limit parameters such as feature dimension, window scale, and gradient calculation method. The relevant settings can be adaptively adjusted according to the specific recognition accuracy requirements and model complexity to maintain the flexibility and generalization ability of the system.

[0060] Regarding step S4, in this embodiment, after completing the multi-dimensional extraction of image features, a feature vector f representing the character image structure information is obtained as the input of the neural network recognition module in this step.

[0061] To achieve category recognition and confidence scoring of characters displayed on the remote control LCD screen, this embodiment constructs a classification and recognition model based on a feedforward neural network, and combines it with a softmax function (Softmax) output mechanism to generate category probability distribution and scoring results.

[0062] Specifically, the neural network model described in this embodiment consists of an input layer, several hidden layers, and an output layer, and adopts a fully connected structure. The network structure can be dynamically set according to the input feature dimension d and the total number of character categories C, and is constructed as follows: The input layer receives the image feature vector f with dimension d; Each hidden layer is transformed using a nonlinear activation function, preferably a ReLU (Rectified Linear Unit) activation function; the output layer dimension is C, each output node corresponds to a character category, and the final output is a set of real numbers.

[0063] For the above output results, the Softmax function is used to normalize them to obtain the predicted probability of each type of character: Among them, P(c i |f) is the probability that the input feature vector f is recognized as the i-th type of character; z j and z iare the output values ​​of the jth and ith neurons in the output layer of the neural network, indicating the activation degree of the category; C is the total number of character categories, indicating the set of all possible characters preset on the remote control LCD screen; e is the base of the natural logarithm, and the Softmax function is used to map a real number vector to a probability distribution.

[0064] According to the above probability output, select the category corresponding to the maximum probability As a result of the recognition: At the same time, the maximum probability value P(c i |f) is output as the recognition confidence score, reflecting the model's credibility in the recognition results.

[0065] In actual implementation, in order to improve the recognition model's adaptability to complex situations such as character deformation, tilt, and occlusion, this embodiment preferably constructs a neural network model through a pre-training plus fine-tuning strategy. Specifically, the training process may include the following: In the initial stage, the model is pre-trained on a standard character image set to obtain the initial weight parameters; Then, supervised fine-tuning training is performed using the sample feature vectors to adapt the model to the unique distribution and morphological characteristics of the remote control screen characters. The cross entropy loss function is used as the optimization objective function during model training, which is as follows: in, is the total loss function value; y i is the one-hot encoding of the true label; P(c i |f) is the predicted probability of the neural network for the i-th category.

[0066] After completing the above training and verifying convergence, the model can be deployed in a character recognition system to achieve real-time recognition and trustworthy scoring of image inputs.

[0067] Furthermore, this embodiment does not specify parameters such as the number of neural network layers, the number of neurons per layer, or the type of activation function. In practice, these parameters can be flexibly configured based on the target hardware platform, the number of characters, and the required model capacity. To ensure the recognition system's responsiveness and embedded adaptability, the trained network can also be optimized through model compression strategies such as pruning, quantization, or distillation.

[0068] In summary, this embodiment achieves accurate recognition of character categories and numerical quantification output of confidence by inputting the foreground image feature vector into a neural network classification model based on Softmax output, providing a standardized and scorable recognition basis for image understanding and task control of the remote control LCD screen.

[0069] For step S5, in this embodiment, after obtaining the character category recognition results and their corresponding confidence scores, a stability judgment mechanism is further introduced to improve the robustness and practical usability of the recognition system, and to avoid instantaneous erroneous recognition caused by image noise, blur, local occlusion, etc. from affecting the overall control process.

[0070] To this end, this embodiment introduces a joint judgment mechanism based on confidence thresholds and historical results to perform dynamic steady-state judgment on the classification results output by the neural network, ensuring that the character categories finally output are stable and continuous, and suitable for subsequent logic control module processing.

[0071] Specifically, let the recognition output of the current image frame be the character category The corresponding confidence score is: in, is the neural network recognition result of the current frame image; f t is the image feature vector corresponding to the current frame; s t : is the maximum category probability value obtained by the Softmax function, which is used as the confidence score of the frame.

[0072] In this embodiment, a fixed or dynamically updated confidence threshold θ is set to judge the reliability of the recognition result. t When θ > θ, the recognition result is considered to have preliminary credibility and can enter the subsequent stability verification process; otherwise, the recognition result of the current frame is marked as "uncertain" and discarded or recorded as an unstable state.

[0073] After passing the preliminary confidence screening, this embodiment introduces a continuous frame consistency verification strategy to improve the stability judgment ability of the recognition results.

[0074] Let the recognition results in the latest T consecutive frames be recorded as sequence The corresponding confidence score is {s t -T+1,s t-T+2 ,…,s t}.

[0075] In this embodiment, the following judgment criteria are preferably adopted: If within this time window, a character category c * If the number of occurrences of exceeds the set ratio (e.g., majority vote), and the corresponding confidence scores are all higher than the threshold θ, then the character category is determined to be “stable output”; If the recognition results of consecutive frames are scattered, or the confidence fluctuates violently, and there is no stable high-confidence output, the recognition result of the current window is considered to be unstable.

[0076] In order to further control the triggering timing of stable output, this embodiment also introduces a state transition mechanism. Specifically: The system maintains an internal state variable to record the previous stable recognition result; When the current recognition result and the continuous frames are judged to be stable, the stable recognition state is updated; If the current output fluctuates but does not meet the stability condition, the previous state is maintained and the system enters the waiting or buffering phase.

[0077] The above mechanism can be formally described as a finite state machine, whose state transition depends on the confidence threshold comparison result and the inter-frame recognition consistency statistics.

[0078] To improve system response efficiency, this embodiment also supports dynamic adjustment of the confidence threshold θ based on the magnitude of input image changes. For example, if a sudden change in the image background or strong noise disturbance is detected, the threshold requirement can be temporarily increased, delaying the output update. Conversely, when the image is continuously stable and the confidence level is high, the judgment criteria can be appropriately relaxed to improve real-time performance.

[0079] In addition, in the above steps, the image change detection module can be implemented based on the image structural entropy calculation method. Structural entropy is a metric to measure the complexity of the grayscale distribution of a local area of ​​an image. The calculation formula is as follows: Among them, H(x,y) is the local structural entropy value of the image near the coordinate (x,y); L is the number of gray levels of the image; p j is the frequency of occurrence of the j-th grayscale value in the sliding window centered at (x, y); δ is a small positive number added to prevent zero values ​​from appearing in the logarithmic calculation; log(·): preferably a natural logarithm function.

[0080] By monitoring the changes in the mean structural entropy of consecutive image frames, it is possible to assist in judging the image change trend, thereby triggering the dynamic adjustment of the confidence judgment mechanism.

[0081] In summary, this embodiment realizes the reliability assessment and state control of the neural network character recognition results by setting the confidence threshold and introducing the continuous frame stability verification mechanism, avoids the misrecognition transmission caused by transient noise, and improves the stability and practicality of the recognition system.

[0082] This stability judgment mechanism not only ensures the output quality of character recognition results, but also provides a reliable data basis for subsequent system interactive control and automated operations.

[0083] In general, this method introduces low-rank sparse decomposition and normalization preprocessing, combines image complexity analysis with structural entropy calculation, cooperates with character recognition and Softmax scoring mechanism based on feedforward neural network, and subsequent confidence threshold judgment and multi-frame stability control process. It achieves robust feature extraction, accurate recognition and dynamic adaptive judgment of result credibility of character images in weak structure and complex background scenes such as embedded LCD screens, thereby providing a more stable and reliable character recognition solution for system-level control and subsequent interactive applications.

[0084] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A remote control LCD screen image detection method based on computer vision, characterized in that: The following steps are involved: S1. Obtain an original image containing the remote control, and perform grayscale conversion and filtering denoising on the original image to obtain a preprocessed image; S2. Based on the preprocessed image, calculate the local information entropy and its change rate of the image by sliding windows at multiple scales, generate a multi-scale entropy gradient heat map, and screen out candidate areas with high entropy differences as LCD display areas based on a set threshold; S3. Vectorize and divide the candidate region into blocks and construct an image matrix. Apply a low-rank and sparse joint decomposition model including a regularization parameter to the matrix to separate the foreground character structure and background interference, thereby obtaining a sparse reconstructed image after character enhancement. S4. Extracting multidimensional features including directional gradient, structural entropy, and grayscale projection from the candidate character region in the sparsely reconstructed image, and inputting the features into a neural network classification model to obtain a character recognition result and its corresponding confidence score; S5. Based on the confidence score and the structural entropy value of the sparsely reconstructed image, a joint scoring function is constructed and a recognition quality score is calculated. The recognition quality score is compared with a preset scoring threshold. Based on the comparison result, the information entropy calculation scale and the sparse modeling regularization parameter are updated and adjusted to achieve adaptive optimization of image region extraction and reconstruction recognition performance.

2. The remote control LCD screen image detection method based on computer vision according to claim 1, characterized in that: In the step S1 , the grayscale conversion includes converting the original image into a grayscale image, and applying Gaussian filtering to the grayscale image to remove noise.

3. The remote control LCD screen image detection method based on computer vision according to claim 1, characterized in that: In step S2, the steps of calculating the local information entropy and its change rate of the image by sliding windows at multiple scales and generating a multi-scale entropy gradient heat map include: Under the preset scale set, a sliding window of corresponding size is used to traverse all pixel areas on the preprocessed image based on each scale, and the grayscale histogram probability distribution of the local image is calculated in each window, and the local information entropy value is calculated accordingly, where the calculation formula of the local information entropy value is: Where L is the grayscale level; (x, y) is the center coordinate of the sliding window; is the local information entropy value; p j is the grayscale histogram probability distribution of the local image; s i For every scale; After obtaining the local information entropy value, the gradient change rate of the entropy value in the spatial position is further calculated at the same scale. The calculation formula is: in, is the scale s i The local information entropy gradient modulus at the coordinate point (x, y); is the gradient of information entropy in the horizontal direction; is the gradient of information entropy in the vertical direction; The gradient maps at each scale are fused according to the preset weights to obtain the final multi-scale entropy gradient heat map. The fusion formula is: Among them, α i is the preset weight and satisfies the normalization condition; T(x,y) is the pixel response value at the coordinate point (x,y) in the final generated multi-scale entropy gradient heat map; n is the number of scales in the scale set; The multi-scale entropy gradient heat map reflects the comprehensive intensity of structural changes of the image at different scales and is used for subsequent candidate region extraction.

4. The remote control LCD screen image detection method based on computer vision according to claim 3, characterized in that: In step S2, the step of selecting a candidate area with a high entropy difference as a liquid crystal display area according to a set threshold comprises: normalizing the response values ​​of all pixel positions in the image based on the generated multi-scale entropy gradient heat map to obtain a normalized response map; The normalized response map is compared with a preset threshold; Based on the comparison results, the connected areas formed by all the pixel coordinate points that meet the requirements are screened out and used as the candidate area set of the LCD display area for subsequent image modeling and recognition processing.

5. The remote control LCD screen image detection method based on computer vision according to claim 1, characterized in that: In step S3, the steps of vectorizing and dividing the candidate area into blocks and constructing an image matrix include: Each candidate region is divided into non-overlapping image sub-blocks of size w×h, and each sub-block is concatenated pixel by pixel in row-first order to generate a vector of dimension 1×(w·h), where the generation formula is: in k =thing(B k ); Among them, B k is the k-th image sub-block, whose dimension is w×h; vec(.) is the vectorized operation in row-priority order; v k is the pixel vector of the kth sub-block; All obtained sub-block vectors are concatenated column by column to form an image observation matrix, which is used for subsequent low-rank and sparse joint decomposition processing to enhance the foreground character structure.

6. The remote control LCD screen image detection method based on computer vision according to claim 1, characterized in that: In step S3, the step of applying a low-rank and sparse joint decomposition model including a regularization parameter to the matrix to separate the foreground character structure and background interference, thereby obtaining a sparse reconstructed image after character enhancement includes: First, the image observation matrix is ​​decomposed through a low-rank and sparse joint decomposition model to obtain the foreground character structure matrix and the background interference matrix; In the process of image observation matrix decomposition, an objective function containing regularization parameters is applied, and the objective function is: Where D is the image observation matrix; L is a low-rank matrix representing the foreground character structure; S is a sparse matrix representing the background interference part; ‖·‖ F is the Frobenius norm of the matrix, which is used to measure the difference between matrices; ‖L‖ * is the nuclear norm of the low-rank matrix L, which promotes low-rank features; ‖S‖1 is the l1 norm of the sparse matrix S, which promotes sparsity; Through the above decomposition, the foreground character structure matrix and background interference matrix are optimized.

7. The remote control LCD screen image detection method based on computer vision according to claim 1, characterized in that: In step S4, the step of extracting multidimensional features including directional gradient, structural entropy and grayscale projection includes: Based on the image restored by the foreground low-rank matrix obtained by the low-rank and sparse joint decomposition model, the following multidimensional feature extraction operations are performed: Extract directional gradient features: The Sobel operator is used to calculate the horizontal gradient and vertical gradient of each pixel in the image, and the gradient magnitude map and gradient direction map are constructed accordingly. Extract structural entropy features: A fixed-size local area is selected with the pixel point as the center, the grayscale probability distribution in the area is statistically analyzed, and the local structural entropy response value of the point is calculated; Extract grayscale projection features: The image is projected with grayscale accumulation along the horizontal direction and the vertical direction respectively to obtain a horizontal projection vector and a vertical projection vector; the directional gradient feature, the structural entropy feature and the grayscale projection feature are obtained to jointly constitute a multi-dimensional feature representation of the image for use in subsequent character positioning or recognition processing.

8. The remote control LCD screen image detection method based on computer vision according to claim 1, characterized in that: In step S4, the step of inputting the features into the neural network classification model to obtain the character recognition results and their corresponding confidence scores includes: The directional gradient features, structural entropy features and grayscale projection features extracted from the image are spliced ​​in a unified order to construct a one-dimensional feature vector as the input of the neural network classification model; The one-dimensional feature vector is input into the pre-trained neural network classification model, and the predicted probability distribution vector of each character category is obtained through the forward propagation process; Based on the predicted probability distribution vector, the character category corresponding to the maximum probability is used as the character recognition result, and the maximum probability value of the character category is used as the confidence score of the recognition result for subsequent screening and decision-making.

9. The remote control LCD screen image detection method based on computer vision according to claim 1, characterized in that: In step S5, the steps of constructing a joint scoring function and calculating the recognition quality score include: Based on the confidence score of the character recognition results and the multi-dimensional structural information of the image region, a joint scoring function is constructed to comprehensively evaluate the recognition quality. The specific steps include: Define the joint scoring function, the function form is: in, is the confidence score of the neural network recognition result; S is the structural stability score of the image region; α and β are the weight coefficients of the joint scoring function; The calculation of the structural stability score S is based on the comprehensive representation of the structural entropy mean and gradient amplitude variance of the image. The calculation rule is: Among them, H(x,y) is the local structural entropy value of the pixel (x,y) in image I; Var(G mag ) is the image gradient magnitude map G mag The global variance of Ω is the pixel set of the character area of ​​the image; N = |Ω| is the number of pixels in the character area; γ∈[0,1] is the fusion coefficient of structural entropy and gradient variance; The joint scoring function represents the overall reliability of the current character recognition result and is used for subsequent screening strategies, result optimization and dynamic threshold control.

10. The remote control LCD screen image detection method based on computer vision according to claim 1, characterized in that: In step S5, the steps of comparing the recognition quality score with a preset score threshold and updating and adjusting the information entropy calculation scale and the sparse modeling regularization parameter based on the comparison result include: Set a scoring threshold, compare the recognition quality score with the threshold, and judge the reliability of the current recognition result. The judgment conditions are: Q≥T Q Reliable identification, Q <T Q Identification is unreliable; If the recognition is unreliable, update and adjust the following two key parameters: Information entropy calculation scale parameters: Set the minimum scale, maximum scale and current scale, and use the adaptive step size strategy to increase the scale to enhance local statistical stability. The update rule is: s t+1 =min(s t +d s ,s max ); Among them, δ s The increment step of the preset scale; s t+1 Calculate the scale of the updated structural entropy; s t is the current scale; s max is the maximum scale; Sparse modeling regularization parameter: The regularization term coefficient is adjusted based on the current score gap to improve the robustness of foreground extraction. The update rule is defined as: l t+1 =λ t ·(1+η·(1-Q)); Among them, λ t is the current regularization parameter; η is the regularization adjustment rate factor; Q is the joint score value; λ t+1 is the updated regularization parameter; If the recognition is reliable, the current parameters are kept unchanged and the subsequent recognition process continues.

Images