Method and system for recognizing definition of characters on computer desktop by using large pattern

Abstract

The invention discloses a method and a system for identifying computer desktop character definition by using a large pattern, and relates to the field of computer desktop character definition identification methods. According to the method and the system for identifying the computer desktop character definition by using the large pattern, a picture sample is selected, a training set is made, then character information is extracted from an image through an OCR (Optical Character Recognition) module, a Transform module is utilized to process the image and evaluate the character definition, and a model is trained by using an optimizer AdamW for the acquired character information; and finally carrying out image quality evaluation through an image calculation function. According to the method and the system for identifying the computer desktop character definition by using the large pattern, an efficient and accurate image character definition identification model is designed by combining OCR and Transform architectures. The OCR module is used for extracting character information in the image, and the Transformer analyzes the image content and the definition of characters through a strong feature learning ability, so that the method is more accurate than a traditional method for analyzing and judging the definition of the characters only by judging the definition of the images.

Description

Technical Field

[0001] The present invention relates to the technical field of computer desktop text clarity recognition methods, and specifically to a method and system for recognizing the clarity of computer desktop text using a large model. Background Art

[0002] Taking screenshots of the computer desktop screen or recording videos of the computer desktop screen is a very common function. However, due to the influence of the environment and operating conditions, there are significant differences in clarity when taking screenshots or recording videos. Currently, the clarity of mainstream software is defined by two physical dimensions, namely resolution and bit rate. However, the definition of these two physical dimensions does not guarantee a strict judgment on whether the text in the computer desktop screen screenshot or recorded video is clearly recognizable.

[0003] Therefore, when taking screenshots of the computer desktop screen or recording videos, it is necessary to dynamically and strictly judge whether the text is clearly recognizable. If it is not clear, the resolution and bit rate of the screenshot and recording need to be increased to ensure that the text is clearly recognizable; if it is too clear, the resolution and bit rate of the screenshot and recording need to be decreased to save storage space and transmission time and reduce costs.

[0004] Currently, mainstream software uses traditional image processing techniques such as grayscale images, binarization, and denoising methods to recognize and judge text clarity.

[0005] With the application of AI technology and large models, a text recognition large model trained with a large amount of computer desktop screen image data is used to strictly judge each frame of data in the computer desktop screen screenshot or recorded video, thereby greatly improving the accuracy of text clarity recognition and strict judgment, obtaining the optimal value, and thus achieving the purpose of dynamically adjusting the resolution and bit rate of the screenshot and recording, saving storage costs and transmission costs. It has great significance for future software applications of computer desktop screen screenshots and computer desktop screen recording videos.

[0006] Currently, the methods for clarity judgment in mainstream software are as follows: In the published patent with the publication number CN110473189A, a method and system for judging the clarity of text images are recorded. This patent provides a method for judging the clarity of text images, including multiple steps such as image preprocessing, binarization processing, and gradient calculation. By combining three evaluation indicators (two ratios in the clarity evaluation value of the binary image and the clarity evaluation value of the gradient image) for judgment, the accuracy of the judgment is improved. This patent uses technologies in the field of image processing, etc., and according to the characteristics of text images, the Canny edge detection algorithm and binarization method are used to extract the text area of the image. The Otsu algorithm is used to obtain the binary image and threshold for the local area, and the threshold increment is set to perform binarization processing again. The number of zero elements in the binary image and the number of changed pixels in the binary image under the action of the threshold increment are counted. The ratios of the two values to the total number of pixels are used as the evaluation values, combined with the evaluation value calculated by the gradient, and the clarity of the image is judged by comparing with the reference threshold.

[0007] In the published patent with the publication number CN116012859A, a method, device, and equipment for judging text image rejection based on clarity indicators are recorded. This technical solution determines multiple clarity indicator values corresponding to the target text image to be judged, as well as the clarity interval thresholds corresponding to each clarity indicator value. According to each clarity indicator value and the clarity interval threshold corresponding to it, the score corresponding to each clarity indicator is determined. According to the scores corresponding to each clarity indicator and the target weights matching each clarity indicator, the quality score corresponding to the target text image is determined. According to the quality score corresponding to the target text image, the rejection judgment result corresponding to the target text image is determined, which can improve the accuracy of the text image rejection judgment result.

[0008] The above methods for judging clarity are all based on image technology and are judged through technical parameters. However, in the case of complex computer desktop software, it is very difficult to predict whether the text can be clearly recognized. Therefore, the large model technology is introduced. The goal is to use a large model to recognize the clarity of the text in the image. The clarity recognition can be achieved by predicting the clarity score or dividing the image into different clarity levels.

[0009] To solve the above problems, a method and system for recognizing the clarity of computer desktop text using a large model are provided. Summary of the Invention

[0010] Aiming at the deficiencies of the prior art, the present invention provides a method and system for recognizing the clarity of computer desktop text using a large model, which solves the problem that the existing methods for judging clarity are all based on image technology and are judged through technical parameters. However, in the case of complex computer desktop software, it is very difficult to predict whether the text can be clearly recognized.

[0011] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for recognizing the clarity of computer desktop text in a large mode, including the following steps; Step 1: Select picture samples, make a training set, and establish an image clarity recognition model at the same time; Step 2: Extract text information from the image through the OCR module; Step 3: The Transformer module processes the evaluation of image and text clarity; Step 4: Use the optimizer AdamW to train the image clarity recognition model with the obtained text information; Step 5: Conduct image quality evaluation through a loss function; Step 6: Optimization and deployment; During the optimization process, compress the large model through knowledge distillation, improve the model inference efficiency, and perform model quantization to reduce the calculation amount and improve the inference speed; During the deployment process, use a deep learning framework for model deployment, export the model in ONNX format, and use the TensorRT tool for acceleration.

[0012] Preferably, the picture samples in Step 1 include multiple groups of computer desktop pictures with different clarity levels. By collecting screenshots of the computer working desktop screen pictures, make a dataset according to different clarity levels, and label the clarity score of each image.

[0013] Preferably, the clarity scores include blurred, clear, and very clear.

[0014] Preferably, in Step 2, the OCR module uses EasyOCR. By inputting an image, EasyOCR automatically recognizes the text area, location information, and text content in the image.

[0015] Preferably, the specific process of the Transformer module includes; Image feature extraction: Use ViT to extract features from the image. By dividing the image into small blocks and performing self-attention calculations on these small blocks, rich visual features in the image are extracted; OCR feature extraction: The text area and location features extracted by OCR are fused with the visual features of the image. The text area information and text blur degree extracted by OCR are used as additional input features and fed into the Transformer network together with the image features; Clarity scoring: Use the fully connected layer of the Transformer model to output the clarity score. Input a classification problem and output multiple clarity levels, including blurred, clear, and very clear, and use cross-entropy loss for classification processing; 。

[0016] Preferably, when training a Transformer-based OCR and image sharpness recognition model, AdamW is used for training. The key steps include: Setting the learning rate: By adjusting the learning rate to balance the convergence speed and the final model accuracy, mapping the image and OCR features to the sharpness score; Training process: Updating the model parameters through backpropagation and the optimizer; Evaluation and tuning: Evaluating the model through the validation set, adjusting the learning rate, and using a learning rate scheduler.

[0017] Preferably, in step five, when training the image quality assessment model, PSNR and SSIM are used as part of the loss function to optimize the model. MSE loss is used to optimize PSNR, and PSNR and SSIM are used as evaluation metrics to monitor the performance of the model.

[0018] Preferably, the PSNR is the ratio of the signal to the noise between two images, used to evaluate the image quality. The higher the value, the better the image quality. Its specific calculation formula is; ; where is the maximum pixel value of the image, and MSE is the mean square error, which measures the difference in pixel values between two images. The calculation formula is: ; where I(i,j) and K(i,j) are the pixel values of the original image and the generated image at the position (i,j) respectively, and M and N are the width and height of the image.

[0019] Preferably, the SSIM is used to measure the similarity in brightness, contrast, and structure between two images, and is suitable for measuring the perceptual quality of images; ; where is the mean value of the two images; is the variance of the two images; is the covariance of the two images; and are constants added to avoid a zero denominator. The value range of SSIM is from -1 to 1. The closer the value is to 1, the better the image quality.

[0020] Preferably, when training an image quality assessment model, PSNR and SSIM can be used as part of the loss function to optimize the model. MSE loss can be used to optimize PSNR, or SSIM can be directly used as the loss function. Both PSNR and SSIM require a real image as a reference image for calculation. Therefore, high-quality reference images need to be ensured during the training process.

[0021] Preferably, a system for recognizing the clarity of computer desktop text using a large pattern recognition includes: A data processing module, where the OCR module is mounted in the data processing module and is used to process picture information to extract text data information in the picture; A data training module, and the data training module is the optimizer AdamW; A data storage module, and the data storage module is used to store the original picture data and model training information; An analysis and evaluation module, and an image calculation function is introduced in the analysis and evaluation module to perform image quality.

[0022] The present invention discloses a method and system for recognizing the clarity of computer desktop text, and the beneficial effects thereof are as follows: 1. The method and system for recognizing the clarity of computer desktop text by using a large pattern recognition design an efficient and accurate image text clarity recognition model by combining OCR and the Transformer architecture. The OCR module is used to extract text information in the image, while the Transformer analyzes the clarity of the image content and text through its powerful feature learning ability, which is more accurate than the traditional method of only relying on judging the clarity of the picture to analyze and judge the clarity of the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0024] Figure 1 It is a flowchart of the method for the clarity of computer desktop text of the present invention; Figure 2 It is a functional framework diagram of the recognition system of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0026] By providing a method and system for recognizing the clarity of computer desktop text in a large mode, the embodiments of the present application solve the problem that existing clarity judgment methods are all based on image technology and judged by technical parameters. However, in the case of complex computer desktop software, it is very difficult to predict whether the text can be clearly recognized.

[0027] To better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0028] The embodiments of the present invention disclose a method for recognizing the clarity of computer desktop text in a large mode. According to the attached Figure 1-2 shown, it includes the following steps; Step 1: Select picture samples, make a training set, and establish an image clarity recognition model at the same time; Step 2: Extract text information from the image through the OCR module; Step 3: The Transformer module processes the evaluation of image and text clarity; Step 4: Use the optimizer AdamW to train the image clarity recognition model with the obtained text information; Step 5: Perform image quality evaluation through a loss function; Step 6: Optimization and deployment; During the optimization process, knowledge distillation is used to compress the large model, improve the model inference efficiency, and perform model quantization to reduce the amount of calculation and improve the inference speed; During the deployment process, a deep learning framework is used for model deployment, the model is exported in the ONNX format, and the TensorRT tool is used for acceleration.

[0029] The picture samples in Step 1 include multiple groups of computer desktop pictures with different clarity levels. By collecting screenshots of the computer working desktop screen pictures, a data set is made according to different clarity levels, and the clarity score of each image is marked.

[0030] The clarity scores include blurred, clear, and very clear.

[0031] In Step 2, the OCR module uses EasyOCR. By inputting the image, EasyOCR automatically recognizes the text area, position information, and text content in the image.

[0032] The specific process of the Transformer module includes: Image feature extraction: Use ViT to extract features from the image. By dividing the image into small patches and performing self-attention calculations on these patches, rich visual features in the image are extracted; OCR feature extraction: The text regions and location features extracted by OCR are fused with the visual features of the image. The text region information and text blur degree extracted by OCR are used as additional input features and fed into the Transformer network together with the image features; Clarity scoring: Use the fully connected layer of the Transformer model to output the clarity score. Input a classification problem and output multiple clarity levels, including blurred, clear, and very clear. Use cross-entropy loss for classification processing; .

[0033] When training the Transformer-based OCR and image clarity recognition models, use AdamW for training. Its key steps include: Set the learning rate: Balance the convergence speed and the final model accuracy by adjusting the learning rate, and map the image and OCR features to the clarity score; Training process: Update the model parameters through backpropagation and the optimizer; Evaluation and tuning: Evaluate the model through the validation set, adjust the learning rate, and use learning rate scheduling.

[0034] In step five, when training the image quality assessment model, use PSNR and SSIM as part of the loss function to optimize the model. Use MSE loss to optimize PSNR, and use PSNR and SSIM as evaluation metrics to monitor the performance of the model.

[0035] PSNR is the ratio of the signal to the noise between two images, used to evaluate the image quality. The higher the value, the better the image quality. Its specific calculation formula is; ; where MAX_I is the maximum pixel value of the image, and MSE is the mean squared error, which measures the difference in pixel values between two images. The calculation formula is: ; where I(i,j) and K(i,j) are the pixel values of the original image and the generated image at position (i,j) respectively, and M and N are the width and height of the image.

[0036] SSIM is used to measure the similarity in brightness, contrast, and structure between two images and is suitable for measuring the perceptual quality of images; ; where is the mean of the two images; is the variance of the two images; is the covariance of the two images; and are constants added to avoid a zero denominator. The value range of SSIM is from -1 to 1. The closer the value is to 1, the better the image quality.

[0037] When training an image quality assessment model, PSNR and SSIM can be used as part of the loss function to optimize the model. Use MSE loss to optimize PSNR, or directly use SSIM as the loss function. At the same time, both PSNR and SSIM require a real image as a reference image for calculation. Therefore, high-quality reference images need to be ensured during the training process.

[0038] In practical applications, PSNR and SSIM are combined into a weighted loss function for model training. By adjusting the weights, the pixel-level error and structural similarity can be balanced.

[0039] Its specific formula is;

[0040] where: α and β are weight coefficients used to control the contributions of SSIM and MSE; MSE is the mean squared error, which is directly related to PSNR, and PSNR is the logarithmic transformation of MSE; 1 - SSIM is because the larger the value of SSIM, the better the quality, while the loss function is usually better when it is smaller. In practical applications, if more attention is paid to structural similarity, α can be increased, such as α = 0.8, β = 0.2; if more attention is paid to pixel-level accuracy, β can be increased, such as α = 0.2, β = 0.8.

[0041] At the same time, when training an image quality assessment model, PSNR and SSIM can be used in stages. In the first stage, PSNR or MSE can be used as a single loss function to quickly converge to pixel-level accuracy; then switch to SSIM or a combined loss function in the second stage to optimize structural similarity and perceptual quality.

[0042] During the training process, monitor the values of PSNR and SSIM simultaneously to comprehensively evaluate the model performance. A comprehensive score can be designed, and this comprehensive score can be a weighted average to guide model selection; Its formula is; Score = γ·PSNR+(1 - γ)SSIM where γ is the weight coefficient used to balance the contributions of PSNR and SSIM.

[0043] According to the attached Figure 2 As shown, a system for recognizing the clarity of computer desktop text using large-scale pattern recognition includes: A data processing module, with an OCR module integrated in the data processing module, used to process picture information to extract the text data information in the picture; A data training module, and the data training module is the optimizer AdamW; A data storage module, which is used to store the original picture data and model training information; An analysis and evaluation module, in which the image quality is analyzed by introducing an image calculation function.

[0044] By combining OCR and the Transformer architecture, an efficient and accurate image text clarity recognition model is designed. The OCR module is used to extract the text information in the image, while the Transformer analyzes the clarity of the image content and text through its powerful feature learning ability, which is more accurate than the traditional method of only relying on judging the clarity of the picture to analyze and judge the clarity of the text.

[0045] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying the clarity of computer desktop text using a large pattern, characterized in that: The steps include: Step 1: Select image samples, create a training set, and establish an image clarity recognition model; Step 2: Extract text information from the image through the OCR module; Step 3: The Transformer module processes the evaluation of image and text clarity; Step 4: Use the optimizer AdamW to train the image clarity recognition model using the acquired text information; Step 5: Evaluate image quality through loss function; Step 6: Optimization and deployment; In the optimization process, knowledge distillation is used to compress the large model, improve the model reasoning efficiency, and perform model quantization to reduce the amount of calculation and improve the reasoning speed; During the deployment process, a deep learning framework is used to deploy the model, the model is exported to ONNX format, and the TensorRT tool is used for acceleration.

2. The method for recognizing the clarity of computer desktop text using a large pattern according to claim 1, characterized in that: The image samples in step 1 include multiple groups of computer desktop images of different resolutions. The image samples are collected by taking screenshots of the computer work desktop, creating data sets of different resolutions, and annotating the resolution score of each image.

3. The method for recognizing the clarity of computer desktop text using a large pattern according to claim 2, characterized in that: The clarity ratings include blurry, clear, and very clear.

4. The method for recognizing the clarity of computer desktop text using a large pattern according to claim 1, characterized in that: In step 2, the OCR module uses EasyOCR, and by inputting an image, EasyOCR automatically recognizes the text area, position information and text content in the image.

5. The method for recognizing the clarity of computer desktop text using a large pattern according to claim 1, characterized in that: The specific process of the Transformer module includes: Image feature extraction: Use ViT to extract features from images by dividing the image into small blocks and performing self-attention calculations on these small blocks to extract rich visual features from the image; OCR extracts features; the text area and position features extracted by OCR are fused with the visual features of the image, and the text area information and text blur degree extracted by OCR are used as additional input features and fed into the Transformer network together with the image features; Clarity score: Use the fully connected layer of the Transformer model to output the clarity score. Input a classification problem and output multiple clarity levels, including blurry, clear, and very clear. Use cross entropy loss for classification. 。 6. The method for recognizing the clarity of computer desktop text using a large pattern according to claim 5, characterized in that: When training the Transformer-based OCR and image clarity recognition model, AdamW is used for training. The key steps include: Setting the learning rate: Balance the convergence speed and final model accuracy by adjusting the learning rate to map image and OCR features to clarity scores; Training process; updating model parameters through back-propagation and optimizer; Evaluation and tuning; evaluate the model through the validation set, adjust the learning rate, and use learning rate scheduling.

7. The method for recognizing the clarity of computer desktop text using a large pattern according to claim 1, characterized in that: In the step 5, when training the image quality assessment model, PSNR and SSIM are used as part of the loss function to optimize the model, MSE loss is used to optimize PSNR, and PSNR and SSIM are used as evaluation indicators to monitor the performance of the model.

8. The method for recognizing the clarity of computer desktop text using a large pattern according to claim 7, characterized in that: The PSNR is the ratio of the signal to noise between two images, which is used to evaluate the image quality. The higher the value, the better the image quality. The specific calculation formula is: ； in is the maximum pixel value of the image, and MSE is the mean square error, which measures the difference in pixel values ​​between two images. The calculation formula is: ； Where I(i,j) and K(i,j) are the pixel values ​​of the original image and the generated image at position (i,j), respectively, and M and N are the width and height of the image.

9. The method for recognizing the clarity of computer desktop text using a large pattern according to claim 7, characterized in that: The SSIM is used to measure the similarity between two images in terms of brightness, contrast and structure, and is suitable for measuring the perceptual quality of an image; ； in is the mean of the two images; is the variance of the two images; is the covariance of the two images; and It is a constant added to avoid the denominator being zero. The value range of SSIM is -1 to 1. The closer the value is to 1, the better the image quality.

10. A system for recognizing the clarity of computer desktop text using a large pattern, applied to the method for recognizing the clarity of computer desktop text using a large pattern as claimed in any one of claims 1 to 9, characterized in that: include; A data processing module, wherein the OCR module is mounted in the data processing module and is used to process the image information to extract the text data information in the image; A data training module, wherein the data training module is an optimizer AdamW; A data storage module, which is used to store original image data and model training information; An analysis and evaluation module is provided, wherein the image quality is evaluated by introducing an image calculation function.

Images