A vision health monitoring method based on mobile terminal equipment shooting tracking comparison

Abstract

The present application relates to the technical field of computer vision, in particular to a vision health monitoring method based on mobile terminal equipment shooting tracking comparison, which comprises the following steps: respectively collecting a large number of left eye, right eye and binocular photos of teenagers, using the latest target detection network yolov10 combined with opencv to perform eye cutting and preprocessing, and uniformly resizing the images; adopting traditional convolutional neural network CNN and Vision Transformers convolutional neural network to extract eye features in the preprocessed images, using binary classification supervised learning to help distinguish myopic eyes and normal eyes; using the cross-validation method to train the model; and applying multiple device screening.The present application designs a computer vision algorithm with high accuracy, can judge whether the eye is myopic and the myopia degree according to the eye image, and realizes the mobile terminal deployment of the algorithm, so that it can be used for real-time detection on mobile devices such as smart phones.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a vision health monitoring method based on shooting, tracking and comparison using mobile terminal devices. Background Technology

[0002] With the increasing severity of myopia among teenagers, rapid and accurate myopia detection methods are becoming increasingly important. Traditional detection methods rely on medical equipment, which is costly and complex to operate. In recent years, with the development of computer vision technology, image-based myopia detection has become a research hotspot. However, existing technologies still have shortcomings in image preprocessing, feature extraction, and model construction, resulting in limited detection accuracy and generalization ability. Furthermore, directly applying ViT to myopia degree segmentation still faces challenges, including limitations in dataset size and insufficient model optimization for this task.

[0003] Inspired by publications such as "Application of Deep Learning-Based Eye Image Analysis in Myopia Detection" and "Recent Advances of VisionTransformers in Medical Image Classification," this technical solution aims to fill the application gap of VisionTransformers (ViT) in the field of myopia detection and promote the development and application of related technologies. Therefore, we propose a vision health monitoring method based on mobile terminal device-based image capture, tracking, and comparison. Summary of the Invention

[0004] The purpose of this invention is to provide a vision health monitoring method based on shooting, tracking and comparison using mobile terminal devices, so as to solve the problems mentioned in the background art.

[0005] To address the aforementioned technical problems, the present invention aims to provide a vision health monitoring method based on mobile terminal device shooting, tracking, and comparison, comprising the following steps:

[0006] S1. Data collection and image preprocessing: A large number of photos of the left eye, right eye and both eyes of teenagers were collected. The latest object detection network YOLOv10 was used in combination with OpenCV to perform eye cropping and preprocessing. The images were also uniformly resized to improve image quality and consistency.

[0007] S2. Model Construction: Traditional convolutional neural networks (CNN) and Vision Transformers convolutional neural networks are used to extract eye features from preprocessed images. Supervised learning with binary classification is used to help distinguish between myopic eyes and normal eyes.

[0008] S3. Model Training and Evaluation: Use cross-validation to train the model to avoid overfitting and improve its generalization ability; during training, adjust model parameters based on the performance on the validation set to optimize model performance; statistically analyze the prediction results during training and evaluate the model's classification performance using multiple parameters.

[0009] S4. Multi-device screening application: The algorithm is embedded into web applications and multi-terminal mobile applications, enabling vision screening anytime, anywhere, and supporting external camera devices to improve image quality.

[0010] As a further improvement to this technical solution, in step S1, the core idea of ​​YOLOv10 is to divide the input image into multiple grid cells. Each grid cell is responsible for predicting a certain number of bounding boxes and the class probabilities of the objects within these bounding boxes. It consists of a feature extraction layer and a prediction layer. The feature extraction layer contains multiple convolutional layers and pooling layers to extract image features. The convolutional layers perform convolution operations by sliding convolutional kernels across the image. The pooling layers are used to reduce the amount of data. The last few layers of the network are prediction layers, which predict the position of the bounding boxes and the class of the objects based on the previously extracted features. For each grid cell, it predicts the coordinates of multiple bounding boxes. ,in and These are the coordinates of the bounding box center relative to the grid cell. and These are the width and height of the bounding box. It also predicts the probability that an object within each bounding box belongs to a different category. ,in Indicates category;

[0011] The specific steps for eye cropping using YOLOv10 and OpenCV include the following:

[0012] S1.1, Haar feature difference calculation assistance: In the process of eye cropping, Haar feature difference calculation is used to help detect edges in the image;

[0013] S1.2 Pruning based on YOLOv10 prediction results:

[0014] First, the image is input into the YOLOv10 network for object detection; the network outputs the bounding box coordinates predicted for each grid cell. and category probability ;

[0015] Then, bounding boxes categorized as "eyes" are selected, and the image is cropped based on the coordinate information of these bounding boxes.

[0016] As a further improvement to this technical solution, in step S1.1, Haar feature difference calculation is used to help detect edges in the image. The specific algorithm is as follows:

[0017] ;

[0018] in, Indicates coordinates The value obtained by Haar feature difference calculation is used to detect edges in the image to help determine the boundaries of the eye. It is used for traversing regions. and coordinate variables; and These represent two different regions in the image, used to calculate the difference to highlight edge features; It is a function of image pixel values, representing the pixel value of the image at a certain coordinate;

[0019] Different regions may contribute differently to feature differencing; introducing a weighting factor can better reflect this difference. The optimized formula is:

[0020] ;

[0021] in, and They are regions and The weighting factors can be obtained from prior knowledge, experience, or through learning.

[0022] Calculating Haar feature differences at different scales can capture features at different levels of the image; by introducing weighting factors and combining multi-scale analysis, the robustness and accuracy of feature difference calculation can be improved; the optimized formula is:

[0023] ;

[0024] in, To represent different scales, and They are respectively in scale The area below, and These are the corresponding weighting factors. In scale Image pixel value function;

[0025] The features of a pixel in an image are often related to the features of its neighboring pixels. Considering neighborhood information when calculating Haar feature differences can improve the accuracy and stability of the calculation; the optimized formula is:

[0026] ;

[0027] in, It is a function representing a pixel. The influence of neighborhood information on its feature values.

[0028] As a further improvement to this technical solution, in step S1, the preprocessing of the cropped eye image includes grayscale conversion, noise reduction, image enhancement, and image normalization; wherein, image noise reduction includes mean filtering, median filtering, and Gaussian filtering; image enhancement includes contrast enhancement (histogram equalization and grayscale stretching) and sharpening; image normalization includes size normalization and grayscale normalization.

[0029] In this process, grayscale conversion determines the weights based on the human eye's sensitivity to different colors. The specific algorithm is as follows:

[0030] ;

[0031] in, This represents the grayscale value of a pixel in the image after grayscale processing. These represent the pixel values ​​of the red, green, and blue channels of a pixel in a color image, respectively; 0.299, 0.587, and 0.114 are the sensitivity ratios of the human eye to red, green, and blue, respectively.

[0032] As a further improvement to this technical solution, Gaussian filtering is used for noise reduction, and the specific algorithm is as follows:

[0033] ;

[0034] in, This indicates the coordinates after Gaussian filtering. Pixel value at; These are the coordinate variables used to traverse the filter kernel; It is a filter kernel The weight value at the coordinates is used to determine how to weight the surrounding pixels; Indicates the original image in coordinates The pixel value at that location is the original pixel value used in the weighted average calculation; This represents the filter kernel used in Gaussian filtering. It is a two-dimensional matrix, and its size and weight distribution determine the filtering effect.

[0035] Introducing adaptive weights to optimize the Gaussian filtering algorithm:

[0036] First, consider local variance: calculate the original image in terms of... variance within the local region centered ;

[0037] Next, adjust the weights: adjust the weights of the filter kernel based on the local variance. ;

[0038] The final optimized formula, i.e., the filtered pixel values, becomes:

[0039] ;

[0040] in, The adjusted weights;

[0041] Furthermore, the Gaussian filtering algorithm is optimized by incorporating multi-scale filtering:

[0042] First, multiple Gaussian filter kernels of different scales are used. For each filter kernel of different scales, the filtered pixel value is calculated according to the original formula.

[0043] Then, weighted fusion is performed: the results of filtering at different scales are weighted and fused. Let the pixel values ​​after filtering at different scales be... ,in Represents the scale, with weights of . The optimized pixel value is:

[0044] ;

[0045] in, .

[0046] As a further improvement to this technical solution, step S2 employs a combination of traditional convolutional neural networks (CNNs) and Vision Transformers convolutional neural networks to construct the model; the traditional convolutional neural network (CNN) includes convolution operations, pooling operations, and fully connected layers; the working principle of convolution is as follows:

[0047] ;

[0048] in, It is the pixel value output by the convolution, representing the coordinates. The new pixel value obtained after the convolution operation; It is the pixel value of the input image, that is, the original eye image in coordinates. Pixel value at; It is a convolution kernel, which is a small matrix used to slide across the image to perform convolution operations. Its size and element values ​​determine the effect of convolution. It is the bias, a constant term used to adjust the position of the convolution result; It is an activation function (such as ReLU) used to introduce nonlinear factors and increase the expressive power of the model;

[0049] The purpose of pooling is to reduce the amount of data while retaining important feature information; common pooling methods include max pooling and average pooling.

[0050] Max pooling selects the maximum value within a small region as the output;

[0051] Average pooling calculates the average value of pixels within a small region as the output;

[0052] The fully connected layer is used to integrate the feature vectors obtained from the previous convolution and pooling operations and map them to the final output dimension; each neuron in it is connected to all neurons in the previous layer.

[0053] As a further improvement to this technical solution, the core of the Vision Transformers convolutional neural network is the self-attention mechanism; it can calculate the correlation between each position in the image and the other positions, thereby better capturing global information.

[0054] The fundamental formula for self-attention is:

[0055] ;

[0056] in, These are the query vector, key vector, and value vector, obtained by performing a linear transformation on the input features, respectively. It is the dimension of the key vector; The function is used to normalize the calculated weights so that their sum is 1.

[0057] Vision Transformers convolutional neural networks also include multilayer perceptrons (MLPs): after the self-attention mechanism, a multilayer perceptron (MLP) is usually added to further process the features; the MLP consists of multiple fully connected layers, which are used to perform nonlinear transformations on the features processed by the self-attention mechanism to increase the expressive power of the model.

[0058] As a further improvement to this technical solution, in step S2, the extracted eye features include at least three aspects: eyeball structure, retinal structure, and eye appearance; wherein:

[0059] Relevant features of the eyeball structure include eyeball shape, sclera pattern, axial length of the eyeball, and the shape and position of the lens;

[0060] Relevant structural features of the retina include retinal vascular distribution and retinal thickness;

[0061] Relevant features of eye appearance include eyelid shape and interocular distance.

[0062] As a further improvement to this technical solution, in step S3, cross-validation is used to train the model, specifically including:

[0063] S3.1, Divide the eye image dataset processed in step S1 into... A subset of roughly equal size (usually) The value can be 5 or 10, here we assume );

[0064] S3.2, proceed Rounds of training and validation: In each round, a subset is selected as the validation set, and the rest... A subset is used as the training set; the model is trained using the training set, and during the training process, the model learns the relationship between the features of the eye images in the training set and the corresponding labels of myopic or normal eyes;

[0065] S3.3 After each round of training, the model is validated using the validation set of the current round to evaluate its performance on data that was not used in the training.

[0066] As a further improvement to this technical solution, in step S3, the parameters used to evaluate the classification performance of the model include at least accuracy, precision, recall, and F1 score, with the F1 score used as the final evaluation criterion; wherein:

[0067] Accuracy is the proportion of samples that are correctly predicted as either positive or negative. The formula is:

[0068] ;

[0069] Precision is the proportion of samples that were predicted as positive and were actually positive, out of the total number of samples predicted as positive. The formula is:

[0070] ;

[0071] It reflects the accuracy of the model's predictions.

[0072] Recall is the proportion of samples that were predicted to be positive and were actually positive, out of the total number of samples that were actually positive. The formula is:

[0073] ;

[0074] It reflects the model's ability to correctly identify positive examples.

[0075] in, This represents the number of samples that were predicted to be positive and were actually positive. This represents the number of samples that were predicted to be positive but were actually negative. This represents the number of samples that were predicted to be negative but were actually positive. This represents the number of samples that were predicted to be negative and were actually negative. This represents the number of samples predicted as positive. This represents the number of samples that are actually positive.

[0076] The F1 score is an evaluation metric that takes into account both precision and recall. The formula is:

[0077] ;

[0078] It provides a more comprehensive evaluation of model performance, balancing the relationship between accuracy and recall.

[0079] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0080] 1. In this vision health monitoring method based on mobile terminal device shooting, tracking and comparison, a highly accurate computer vision algorithm is developed, which can determine whether the eyes are myopic and the degree of myopia based on eye images; the algorithm is deployed on mobile devices, enabling real-time detection on smartphones and other mobile devices; and a user-friendly interface is provided, allowing ordinary users to easily use the application to self-assess their myopia.

[0081] 2. In this vision health monitoring method based on mobile terminal device shooting and tracking comparison, eye photos taken by mobile phone are used to study a classification model based on the Vision Transformers network structure through computer vision-related deep learning technology. This model is used to determine whether myopia is present and to distinguish the degree of myopia. Currently, the highest accuracy rate of 72% has been achieved in binary classification tasks (determining whether myopia is present) on existing data. Attached Figure Description

[0082] Figure 1 This is an exemplary overall method flowchart of the present invention. Detailed Implementation

[0083] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0084] Example 1

[0085] like Figure 1 As shown, this embodiment provides a vision health monitoring method based on mobile terminal device shooting, tracking and comparison, including the following steps:

[0086] S1. Data collection and image preprocessing: A large number of photos of the left eye, right eye and both eyes of teenagers were collected. The latest object detection network YOLOv10 was used in combination with OpenCV to perform eye cropping and preprocessing. The images were also uniformly resized to improve image quality and consistency.

[0087] Specifically, the collection of eye photographs is targeted at teenagers. This is because teenagers are a high-risk group for myopia, and their eye photographs are easier to distinguish between myopic and normal eyes. Collection requirements: Photos of the left eye, right eye, and both eyes must be collected. During the collection process, it is essential to ensure that the photos clearly reflect the eye characteristics. For example, the subject should look directly at the camera, maintain even lighting on their face, and avoid shadows obscuring the eyes. Simultaneously, the subject should be instructed to keep their head stable to prevent blurry photos due to movement.

[0088] Meanwhile, YOLOv10 boasts advanced object detection capabilities, accurately identifying the eye region in images. OpenCV provides a rich set of image processing functions, which helps to further optimize the cropping process.

[0089] In this step, YOLOv10 is an advanced object detection network based on a deep convolutional neural network architecture. Its core idea is to divide the input image into multiple grid cells, each responsible for predicting a certain number of bounding boxes and the probability of object class within those bounding boxes. It consists of a feature extraction layer and a prediction layer. The feature extraction layer contains multiple convolutional and pooling layers to extract image features. The convolutional layers perform convolution operations by sliding convolutional kernels across the image. Pooling layers reduce the amount of data; common methods include max pooling and average pooling. For example, max pooling selects the maximum value within a small region as the output. The last few layers of the network are prediction layers, which predict the location of the bounding boxes and the object class based on the extracted features. For each grid cell, it predicts the coordinates of multiple bounding boxes. ,in and These are the coordinates of the bounding box center relative to the grid cell. and These are the width and height of the bounding box. It also predicts the probability that an object within each bounding box belongs to a different category. ,in Indicates category;

[0090] The specific steps for eye cropping using YOLOv10 and OpenCV include the following:

[0091] S1.1, Haar feature difference calculation assistance (OpenCV related): In the process of eye cropping, Haar feature difference calculation is used to help detect edges in the image;

[0092] S1.2 Pruning based on YOLOv10 prediction results:

[0093] First, the image is input into the YOLOv10 network for object detection; the network outputs the bounding box coordinates predicted for each grid cell. and category probability ;

[0094] Then, bounding boxes categorized as "eye" are selected, and the image is cropped based on the coordinates of these bounding boxes. For example, if the coordinates of a bounding box are... The cropped eye image area is That is, to extract the top left corner coordinates from the image. The coordinates of the lower right corner are The rectangular region is used as the eye image.

[0095] Further optimization is also included: to improve the accuracy of cropping, some post-processing can be performed on the cropped eye image. For example, the cropped region can be fine-tuned based on the shape features of the eye, or some potentially interfering edges can be removed. This may involve some morphological operation-based algorithms, such as erosion and dilation operations (common image processing operations in OpenCV). Erosion can remove small objects in the image, and dilation can fill small holes in the image. Through these operations, the cropped eye image can be made cleaner and more accurate, which is more conducive to subsequent image preprocessing and model building.

[0096] Furthermore, in step S1.1, Haar feature difference calculation is used to help detect edges in the image. The specific algorithm is as follows:

[0097] ;

[0098] in, Indicates coordinates The value obtained by Haar feature difference calculation is used to detect edges in the image to help determine the boundaries of the eye. It is used for traversing regions. and coordinate variables; and These represent two different regions in the image, used to calculate the difference to highlight edge features; It is a function of image pixel values, representing the pixel value of the image at a certain coordinate; for example... Indicates coordinates Pixel value at;

[0099] Different regions may contribute differently to feature differencing; introducing weighting factors can better reflect this difference. For example, key regions or regions with specific characteristics can be assigned larger weights to highlight their importance in feature differencing calculation. The optimized formula is:

[0100] ;

[0101] in, and They are regions and The weighting factors can be derived from prior knowledge, experience, or through learning. For example, the average importance of different regions can be determined by analyzing a large number of samples, thereby setting corresponding weighting factors.

[0102] Calculating Haar feature differences at different scales can capture features at different levels of an image. For example, at smaller scales, attention can be paid to detailed information; at larger scales, the overall structure and contours of the image can be captured. Introducing weighting factors and combining multi-scale analysis can improve the robustness and accuracy of feature difference calculation; the optimized formula is:

[0103] ;

[0104] in, To represent different scales, and They are respectively in scale The area below, and These are the corresponding weighting factors. In scale The image pixel value function is used. In actual calculations, the original image can be downsampled at different scales first, then the feature difference can be calculated at each scale, and the results from different scales can be fused.

[0105] The features of a pixel in an image are often related to the features of its neighboring pixels. Considering neighborhood information can improve the accuracy and stability of Haar feature difference calculations. For example, the features of pixels within a certain range around the center pixel can be weighted and averaged, and used as the feature value of the center pixel in the feature difference calculation.

[0106] The optimized formula is:

[0107] ;

[0108] in, It is a function representing a pixel. The influence of neighborhood information on its eigenvalues. The neighborhood information function can be designed according to specific needs, such as using a Gaussian function or a mean function.

[0109] The above optimization methods can improve the performance of the Haar feature difference calculation formula, making it more accurate and effective in tasks such as image feature extraction and edge detection. In practical applications, appropriate optimization methods can be selected and adjusted as needed based on the specific problem and data characteristics.

[0110] Further, in step S1, the preprocessing of the cropped eye image includes grayscale conversion, noise reduction, image enhancement, and image normalization; wherein, image noise reduction includes mean filtering, median filtering, and Gaussian filtering; image enhancement includes contrast enhancement (including histogram equalization and grayscale stretching) and sharpening; image normalization includes size normalization and grayscale normalization.

[0111] In this process, grayscale conversion determines the weights based on the human eye's sensitivity to different colors. The specific algorithm is as follows:

[0112] ;

[0113] in, This represents the grayscale value of a pixel in the image after grayscale processing. These represent the pixel values ​​of the red, green, and blue channels of a pixel in a color image, respectively; 0.299, 0.587, and 0.114 are the sensitivity ratios of the human eye to red, green, and blue, respectively.

[0114] Specifically, grayscale conversion is the process of converting a color image into a grayscale image. This is based on the fact that the human eye has different sensitivities to different colors; the human eye is most sensitive to green and least sensitive to blue. This formula is used to convert a color image into a grayscale image. A grayscale image contains only one channel of information. Compared to the three channels of a color image, grayscale processing reduces the amount of data in the image. At the same time, grayscale images can more clearly highlight the structure and texture of the image, emphasizing the structural features of the eye, making subsequent feature extraction and analysis easier.

[0115] Specifically, Gaussian filtering is used for noise reduction, and the specific algorithm is as follows:

[0116] ;

[0117] in, This indicates the coordinates after Gaussian filtering. Pixel value at; These are the coordinate variables used to traverse the filter kernel; It is a filter kernel The weight values ​​at the coordinates are used to determine how to weight surrounding pixels. Different filter kernel settings will result in different value distributions. Indicates the original image in coordinates The pixel value at that location is the original pixel value used in the weighted average calculation; This represents the filter kernel used in Gaussian filtering. It is a two-dimensional matrix, and its size and weight distribution determine the filtering effect. For example, common filter kernel matrices are 3x3 or 5x5, and their internal elements are... The value of .

[0118] Introducing adaptive weights to optimize the Gaussian filtering algorithm:

[0119] First, consider local variance: calculate the original image in terms of... variance within the local region centered Regions with high variance typically indicate that the image has more detail or more noise, requiring a smaller filtering strength; regions with low variance may be smoother regions, allowing for a larger filtering strength.

[0120] Next, adjust the weights: adjust the weights of the filter kernel based on the local variance. A function can be used. To calculate the adjusted weights, for example ,in It is a pre-set maximum variance value. The adjusted weights are... .

[0121] The final optimized formula, i.e., the filtered pixel values, becomes:

[0122] ;

[0123] in, The adjusted weights;

[0124] Furthermore, the Gaussian filtering algorithm is optimized by incorporating multi-scale filtering:

[0125] First, multiple Gaussian filter kernels of different scales are used, such as small-scale, medium-scale, and large-scale filter kernels. For each scale filter kernel, the filtered pixel value is calculated according to the original formula;

[0126] Then, weighted fusion is performed: the results of filtering at different scales are weighted and fused. Let the pixel values ​​after filtering at different scales be... ,in Represents the scale, with weights of . The optimized pixel value is:

[0127] ;

[0128] in, Weights can be determined based on experience or through learning, for example, by analyzing local features of an image or by training a small neural network.

[0129] Through the above optimizations, Gaussian filtering can be made more adaptable to the local features of an image, better preserving image details while removing noise.

[0130] S2. Model Construction: Traditional convolutional neural networks (CNN) and Vision Transformers convolutional neural networks are used to extract eye features from preprocessed images. Supervised learning with binary classification is used to help distinguish between myopic eyes and normal eyes.

[0131] In this step, a model is constructed by combining a traditional Convolutional Neural Network (CNN) and a Vision Transformers CNN. These two network structures each have their advantages: CNN excels in local feature extraction, effectively capturing local texture and detail information in images; while Vision Transformers are adept at processing global information, establishing long-range dependencies, and better understanding the overall structure and semantics of images. By combining the two, features of eye images can be extracted more comprehensively, enabling better differentiation between myopic and normal eyes. A traditional Convolutional Neural Network (CNN) includes convolution operations, pooling operations, and fully connected layers; the working principle of convolution is as follows:

[0132] ;

[0133] in, It is the pixel value output by the convolution, representing the coordinates. The new pixel value obtained after the convolution operation; It is the pixel value of the input image, that is, the original eye image in coordinates. Pixel value at; It is a convolution kernel, which is a small matrix used to slide across the image to perform convolution operations. Its size and element values ​​determine the effect of convolution. It is the bias, a constant term used to adjust the position of the convolution result; It is an activation function (such as ReLU) used to introduce nonlinear factors and increase the expressive power of the model. The ReLU function is defined as follows: That is, when hour, ;then , .

[0134] The purpose of pooling operations is to reduce the amount of data while retaining important feature information. Common pooling methods include max pooling and average pooling: max pooling selects the maximum value within a small region as the output, for example, within a... Within a given area, the maximum value among four pixel values ​​is selected; average pooling, on the other hand, calculates the average value of pixel values ​​within a small area as the output.

[0135] The fully connected layer is used to integrate the feature vectors obtained from the previous convolution and pooling operations and map them to the final output dimension; each neuron in it is connected to all neurons in the previous layer.

[0136] Furthermore, the core of the Vision Transformers convolutional neural network is the self-attention mechanism; it can calculate the correlation between each location in the image and the rest of the locations, thereby better capturing global information.

[0137] The fundamental formula for self-attention is:

[0138] ;

[0139] in, These are the query vector, key vector, and value vector, obtained by performing a linear transformation on the input features, respectively. It is the dimension of the key vector; The function is used to normalize the calculated weights so that their sum is 1.

[0140] Through the self-attention mechanism, the model can dynamically adjust the weights based on the relevance of different locations in the image, thereby better focusing on important feature regions.

[0141] Vision Transformers convolutional neural networks also include multilayer perceptrons (MLPs): after the self-attention mechanism, a multilayer perceptron (MLP) is usually added to further process the features; the MLP consists of multiple fully connected layers, which are used to perform nonlinear transformations on the features processed by the self-attention mechanism to increase the expressive power of the model.

[0142] Specifically, the combination of traditional convolutional neural networks (CNNs) and Vision Transformers convolutional neural networks includes:

[0143] The first type is feature fusion, which includes:

[0144] (a) Early Fusion: Fusion is performed at lower layers of the network. First, a CNN is used to perform preliminary feature extraction on the input image, resulting in a set of local feature maps. Then, these feature maps are input into the encoder of the Vision Transformers, combining with the Transformer's self-attention mechanism to jointly extract higher-level features. During this process, fusion algorithms such as concatenation or weighted summation can be used to fuse the features extracted by the CNN and the Transformer. For example, the feature maps extracted by the CNN and the feature vectors output by the Transformer encoder can be concatenated and then input into subsequent network layers for further processing.

[0145] (ii) Late-stage fusion: Fusion is performed at higher layers of the network. CNNs and Vision Transformers are used independently to extract features from the image, resulting in two different sets of feature representations. Then, in the later stages of the network, these two sets of features are fused. Various fusion strategies can be used, such as feature weighting or multilayer perceptron (MLP) fusion. For example, an MLP can be used to learn how to fuse the features extracted by CNNs and Vision Transformers; the formula can be expressed as follows: ,in These are the characteristics after fusion. These are features extracted by CNN. These are features extracted by Vision Transformers. MLP is a multilayer perceptron that fuses input features by learning weight parameters.

[0146] The second type is a hybrid architecture, which includes:

[0147] (a) Constructing a Hybrid Network: Design a hybrid network architecture that combines CNN and Vision Transformer modules. For example, a CNN module can be used at the front end of the network for local feature extraction, then a Vision Transformer module can be introduced in the middle layer to process global information, and finally a fully connected layer or other classifier can be used at the back end of the network for classification. In this hybrid network, the ratio and connection method of CNN and Vision Transformers can be flexibly adjusted according to specific task requirements and data characteristics.

[0148] (ii) Attention Mechanism Fusion: Combining the attention mechanism of CNNs with the self-attention mechanism of Vision Transformers. For example, a Transformer-like self-attention module can be introduced into CNNs, or the convolutional attention mechanism of CNNs can be used in Vision Transformers to enhance the attention to local features. This can further improve the model's ability to extract features at different levels, while strengthening the interaction between local and global information.

[0149] Furthermore, in step S2, the extracted eye features include at least three aspects: eyeball structure, retinal structure, and eye appearance; wherein:

[0150] Relevant features of the eyeball structure include eyeball shape, sclera pattern, axial length of the eyeball, and the shape and position of the lens;

[0151] Relevant structural features of the retina include retinal vascular distribution and retinal thickness;

[0152] Relevant features of eye appearance include eyelid shape and interocular distance. Specific features are as follows:

[0153] 1. Axial length of the eyeball: Myopia is usually caused by an excessively long axial length of the eyeball. Although the axial length of the eyeball cannot be directly and accurately measured in an eye image, it can be reflected by some indirect features. For example, in an eye image taken from the side, the eyeball of a myopic person may appear more prominent, which may suggest that its axial length is relatively long.

[0154] II. Lens-Related Characteristics: The shape and position of the lens may change in myopic eyes. In a normal eye, the lens can adjust its focus normally, allowing the image to be clearly focused on the retina. However, in a myopic eye, the lens may become more convex, and its accommodative ability may be affected. By analyzing eye images, characteristics such as changes in grayscale values ​​and the sharpness of the lens region can be observed to distinguish between myopic and normal eyes. For example, if the grayscale values ​​of the lens region are relatively uniform and the boundaries are clear, it may be more likely to be a normal eye; if there are abnormal changes in grayscale values ​​and blurred boundaries, it may indicate the possibility of myopia.

[0155] III. Retinal Vessel Distribution: The distribution of blood vessels on the retina may differ between myopic and normal eyes. In normal eyes, the retinal vessels are relatively regular in distribution and uniform in thickness. However, in myopic eyes, due to changes in the eyeball structure, retinal vessels may be stretched or compressed, resulting in distorted, thinned, or locally altered vessel density. Morphological analysis of retinal vessels in eye images can serve as a reference feature for distinguishing between myopic and normal eyes.

[0156] IV. Retinal Thickness: The retinal thickness of a myopic eye may differ from that of a normal eye. Long-term myopia can cause the retina to thin, especially in critical areas such as the macula. Advanced ocular imaging techniques or depth analysis of eye images can be used to obtain information related to retinal thickness, which can help differentiate between myopic and normal eyes.

[0157] V. Eyelid Morphology: Although eyelid morphology is not directly causally related to myopia, some studies have found subtle morphological changes in the eyelids of myopic patients. For example, the upper eyelids of myopic patients may be relatively loose, or the eyelid folds may have different appearances. These characteristics can serve as supplementary information, combined with other ocular features, to distinguish between myopic and normal eyes.

[0158] 6. Interocular Distance: Interocular distance may also differ between myopic and normal eyes. Although this difference may not be very significant, large-scale statistical analyses may reveal that the average interocular distance in myopic patients differs from that in normal eyes. Relevant parameters of interocular distance can be measured in eye images as a potential distinguishing feature.

[0159] S3. Model Training and Evaluation: Use cross-validation to train the model to avoid overfitting and improve its generalization ability; during training, adjust model parameters based on the performance on the validation set to optimize model performance; statistically analyze the prediction results during training and evaluate the model's classification performance using multiple parameters.

[0160] The primary reason for choosing cross-validation to train the model is to avoid overfitting. With limited training data, the model can easily overlearn features from the training data, leading to a decline in its generalization ability on new data. Cross-validation divides the dataset into multiple subsets, using one subset as the validation set and the others as the training set in turn, repeating the training and validation process multiple times. This allows for a more comprehensive evaluation of the model's performance and improves its generalization ability.

[0161] In this step, cross-validation is used to train the model, specifically including:

[0162] S3.1, Divide the eye image dataset processed in step S1 into... A subset of roughly equal size (usually) The value can be 5 or 10, here we assume );

[0163] S3.2, proceed Rounds of training and validation: In each round, a subset is selected as the validation set, and the rest... A subset is used as the training set; for example, in the first round, the first subset is used as the validation set, and the second to fifth subsets are used as the training set; the training set is used to train the model, and during the training process, the model learns the relationship between the features of the eye images in the training set and the corresponding myopic or normal eye labels;

[0164] S3.3 After each round of training, the model is validated using the validation set of the current round to evaluate its performance on data that was not used in the training.

[0165] Furthermore, when adjusting model parameters:

[0166] First, adjust based on validation set performance: During training, closely monitor the performance on the validation set. Validation set performance metrics (such as precision, recall, F1 score, etc.) reflect the model's generalization ability on unseen data. If the validation set accuracy starts to decline, or other performance metrics become unsatisfactory, it indicates that the model may be overfitting.

[0167] Secondly, the parameters and methods for adjustment: Common model parameters that need adjustment include the size and number of convolutional kernels in a convolutional neural network, the number of layers in the network, and the learning rate. For example, if the model's accuracy on the validation set is found to be low, you can try reducing the learning rate to allow the model to learn the features in the training data more slowly, avoiding overfitting. Alternatively, you can increase the number of layers in the neural network to improve the model's expressive power, but at the same time, you should be careful to prevent the overfitting from worsening. If the model is found to be overfitting, you can try adding regularization terms, such as L1 or L2 regularization, to limit the size of the model parameters and prevent the model from becoming too complex.

[0168] Furthermore, in step S3, the parameters used to evaluate the classification performance of the model include at least accuracy, precision, recall, and F1 score, with the F1 score used as the final evaluation criterion; wherein:

[0169] Accuracy is the proportion of samples that are correctly predicted as either positive or negative. The formula is:

[0170] ;

[0171] Precision is the proportion of samples that were predicted as positive and were actually positive, out of the total number of samples predicted as positive. The formula is:

[0172] ;

[0173] It reflects the accuracy of the model's predictions.

[0174] Recall is the proportion of samples that were predicted to be positive and were actually positive, out of the total number of samples that were actually positive. The formula is:

[0175] ;

[0176] It reflects the model's ability to correctly identify positive examples.

[0177] in, This represents the number of samples that were predicted to be positive and were actually positive. This represents the number of samples that were predicted to be positive but were actually negative. This represents the number of samples that were predicted to be negative but were actually positive. This represents the number of samples that were predicted to be negative and were actually negative. This represents the number of samples predicted as positive. This represents the number of samples that are actually positive.

[0178] The F1 score is an evaluation metric that takes into account both precision and recall. The formula is:

[0179] ;

[0180] It provides a more comprehensive evaluation of model performance, balancing the relationship between accuracy and recall.

[0181] S4. Multi-device screening application: The algorithm is embedded into web applications and multi-terminal mobile applications, enabling vision screening anytime, anywhere, and supporting external camera devices to improve image quality.

[0182] Specifically, the model application includes the following steps:

[0183] A1. Mobile terminal device adaptation

[0184] Installation and Setup: A dedicated mobile application has been developed to support major mobile operating systems such as iOS and Android. Users can download and install the application from app stores. After installation, the application will guide users through necessary settings, such as granting access to the camera and allowing storage permissions, to ensure that the application can function properly and save captured eye images.

[0185] Image Acquisition Guidance: The application interface provides clear image acquisition guidance. For example, it can demonstrate the correct shooting posture and distance using a combination of text and images to ensure users can accurately capture clear images of the eyes. Animated demonstrations or video tutorials can be used to help users better understand the shooting requirements. Additionally, the application can set autofocus and exposure adjustment functions to adapt to different shooting environments and improve image quality.

[0186] A2. Eye Image Capture and Upload

[0187] Shooting Process: After opening the application, the user enters the image shooting interface. The application automatically activates the mobile device's camera function and displays a real-time preview on the screen. Following the instructions, the user points the camera at their eyes, adjusting the distance and angle to ensure the eyes are fully in the frame and clearly visible. The application can provide shooting guidelines or frames to help users better position their eyes. After shooting, the user can view the image and retake it if unsatisfied.

[0188] Image Upload: Once the eye image is captured, it will be automatically saved to the local storage of the mobile device. Users can choose to upload the image immediately for analysis or upload it at a later, more suitable time. During the upload process, the application will display an upload progress bar and ensure the secure transmission of image data. Encryption technology and data compression methods can be used to improve upload speed and data security. Simultaneously, the application will perform a preliminary quality check on the uploaded image, such as checking for clarity and completeness. If the image quality does not meet requirements, the application will prompt the user to retake the image or adjust the uploaded image.

[0189] A3. Display of Vision Monitoring Results

[0190] Real-time analysis and feedback: Once the image is successfully uploaded, the server-side vision monitoring model immediately analyzes it. During the analysis, the model extracts features from the eye image and compares them with a pre-trained model to determine whether the user's eye is nearsighted or normal. The analysis results are returned to the mobile device within a short time and displayed to the user on the application interface. The results can be displayed in a concise and clear manner, such as with text descriptions ("nearsighted" or "normal eye") or icons (e.g., glasses icon for nearsightedness, clear eye icon for normal eye).

[0191] Detailed Report Generation: In addition to simple results display, the application can generate detailed vision monitoring reports. These reports can include more information, such as eye features extracted during the analysis, comparisons with features of normal or nearsighted eyes, and warnings of potential vision problems. The reports can be presented in PDF or web page format for easy saving and sharing. Furthermore, the reports provide suggestions such as regular eye exams, attention to eye hygiene, and reasonable time management for screen time to help users better protect their vision.

[0192] It also includes results analysis, including the following steps:

[0193] B1. Accuracy Assessment

[0194] Comparative Validation: To evaluate the model's accuracy, the model's analysis results are compared with the diagnostic results of professional ophthalmologists. A certain number of eye image samples can be collected, and both the model and ophthalmologists analyze and judge them. The results of the two are compared, and evaluation metrics such as accuracy, recall, and F1 score are calculated. If the model's accuracy is not high, the model parameters can be further optimized, the amount of training data increased, and image preprocessing methods improved to enhance the model's performance.

[0195] Cross-validation: Cross-validation involves dividing the dataset into multiple subsets, using one subset as the validation set and the remaining subsets as the training set, and then training and validating the model multiple times. Cross-validation allows for a more comprehensive evaluation of the model's generalization ability and stability. If the model's performance varies significantly across different subsets, it indicates potential overfitting or underfitting, requiring further adjustments to the model's structure and parameters.

[0196] B2. User Feedback Collection

[0197] Establish feedback channels: Set up user feedback channels in the mobile application, such as feedback buttons and online surveys. Users can use these channels to provide developers with feedback on their user experience, their opinions on the accuracy of results, and suggestions for improvement. Developers can regularly collect and analyze user feedback to understand user needs and problems, enabling timely improvements and optimizations to the application.

[0198] Feedback Analysis and Application: Classify and analyze user feedback to extract valuable information. For example, if many user feedback results are inaccurate, further investigation can be conducted to determine the cause, which could be due to image capture quality issues, model performance problems, or improper user operation. Based on the feedback analysis results, corresponding improvement measures can be taken, such as optimizing image acquisition guidance, improving model accuracy, and refining the user interface. Simultaneously, timely responses to user feedback demonstrate the developer's attention and importance, thereby increasing user satisfaction.

[0199] B3. Continuous Improvement and Optimization

[0200] Data Updates and Model Optimization: Over time, new eye image data is continuously collected and added to the training set for ongoing optimization and improvement of the model. The model can be retrained periodically to adjust parameters and adapt to changing real-world conditions. Simultaneously, the latest research findings and technological advancements in ophthalmology are monitored, and new feature extraction methods and model structures are promptly introduced to enhance performance and accuracy.

[0201] Application Functionality Expansion: Continuously expand the functionality of the mobile application based on user needs and market feedback. For example, a vision change trend analysis function can be added, allowing users to track their vision changes over a long period; or an eye health knowledge popularization module can be added, providing more information on eye hygiene, myopia prevention, and other related topics; collaborations with professional ophthalmology institutions can also be undertaken to provide users with online consultations, appointment scheduling, and other services, enhancing the app's usability and value.

[0202] In summary, through data collection and application testing, the aforementioned adolescent myopia detection method based on the combination of hybrid convolutional neural networks and VisionTransformer comprehensively measures the model's classification performance on the test set by calculating evaluation indicators such as accuracy, precision, and recall. The accuracy is 12% higher than that of traditional CNN models, while the precision and recall are 10% and 15% higher, respectively; and the testing speed is more than 80% faster than traditional myopia scale tests.

[0203] Those skilled in the art will understand that the process of implementing all or part of the steps of the above embodiments can be carried out by hardware or by a program instructing the relevant hardware.

[0204] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for monitoring vision health based on mobile terminal device photograph tracking contrast, characterized in that, Comprise the following steps: S1, data collection and image preprocessing: a large number of left eye, right eye and binocular photos of adolescents are collected respectively, the latest target detection network yolov10 is combined with opencv to perform eye cutting and preprocessing, and the images are uniformly resized; S2, model construction: traditional convolutional neural network CNN and Vision Transformers convolutional neural network are used to extract eye features in the preprocessed images, and supervised learning of binary classification is used to help distinguish myopic eyes and normal eyes; Among them, the extracted eye features at least include related features of eye structure, retina structure and eye appearance; wherein: The related features of the eyeball structure include eyeball shape, eye white pattern, eyeball axis length, shape and position of the lens; The related features of the retina structure include retinal blood vessel distribution and retinal thickness; The related features of the eye appearance include eyelid shape and eye distance; S3, model training and evaluation: the model is trained using the cross-validation method; during the training process, the model parameters are adjusted according to the performance of the validation set to optimize the model performance; the prediction results of the training process are counted, and the classification performance of the model is evaluated through multiple parameters; S4, multiple device screening application: the model constructed, trained and evaluated in steps S2-S3 is implanted into web application and multi-terminal mobile application, and supports external camera devices to improve image quality.

2. The method of claim 1, wherein the method is based on mobile terminal device photograph tracking contrast. In the step S1, the core idea of yolov10 is to divide the input image into multiple grid units, each of which is responsible for predicting a certain number of bounding boxes and the class probability of the target in the bounding box; it is composed of a feature extraction layer and a prediction layer; the feature extraction layer contains multiple convolutional layers and pooling layers for extracting image features; the convolutional layer slides on the image through the convolution kernel for convolution operation; the pooling layer is used to reduce the data amount; the last few layers of the network are the prediction layer, which predicts the position of the bounding box and the category of the target according to the extracted features; for each grid unit, it will predict the coordinates of multiple bounding boxes where and are the coordinates of the center of the bounding box relative to the grid unit, and are the width and height of the bounding box; at the same time, the probability of the target in each bounding box belonging to different categories is also predicted where represents the category; The yolov10 combined with opencv for eye cutting specifically comprises the following: S1.1, Haar feature difference calculation assistance: in the eye cutting process, Haar feature difference calculation is used to help detect the edges in the image; S1.2, cutting based on yolov10 prediction results: First, the image is input into the yolov10 network for target detection; the network will output the predicted bounding box coordinates of each grid unit and class probability ; Then, the boundary box of the eye class is selected, and the image is cut according to the coordinate information of the boundary box.

3. The method of claim 2, wherein the method further comprises: determining a contrast sensitivity function of the user based on the captured images. In the step S1.1, Haar feature difference calculation is used to help detect the edges in the image, and the specific algorithm is: ； wherein, represents a value obtained by Haar feature difference calculation at coordinates , which is used to detect edges in the image to help determine the boundary of the eye; is a coordinate variable for traversing the region and ; and represent two different regions in the image, respectively, for calculating differences to highlight edge features; is a function of the pixel value of the image, representing the pixel value of the image at a certain coordinate; Different regions may have different contributions to feature difference, and the introduction of a weight factor can better reflect this difference; the optimized formula is: ； wherein, and are weight factors for the regions and respectively; the weight factors are obtained from prior knowledge, experience or by learning; Calculating Haar feature difference at different scales can capture different levels of features in the image; on the basis of introducing the weight factor, combined with multi-scale analysis, the robustness and accuracy of feature difference calculation can be improved; the optimized formula is: ； wherein, denote different scales, and are regions at scales , and are corresponding weight factors, is an image pixel value function at scale . The features of a pixel point in the image are often related to the features of its neighborhood pixel points; when calculating Haar feature difference, considering the neighborhood information can improve the accuracy and stability of the calculation; the optimized formula is: ； wherein, is a function representing the influence of the neighborhood information of the pixel point on its eigenvalue. 4.The mobile terminal device based vision health monitoring method using the comparison of the tracking of the photographing according to claim 1, wherein: In the step S1, the preprocessed eye images are subjected to preprocessing including grayscale, noise reduction, image enhancement and image normalization; wherein, image noise reduction includes mean filtering, median filtering and Gaussian filtering; image enhancement includes contrast enhancement and sharpening; image normalization includes size normalization and grayscale normalization; The grayscale is determined according to the sensitivity of the human eye to different colors, and the specific algorithm is: ； wherein, represents the gray value of the image after the gray processing at a certain pixel point; respectively represent the pixel values of the red, green and blue channels of a certain pixel point in the color image; 0.299, 0.587 and 0.114 are respectively the sensitivity ratio of the human eye to red, green and blue.

5. The method of claim 4, wherein the method further comprises: determining a contrast sensitivity function of the user based on the captured images. The Gaussian filter is used for noise reduction, and the specific algorithm is: ； wherein, represents the pixel value at the coordinate after Gaussian filtering processing; is a coordinate variable for traversing the filter kernel; is the filter kernel at the coordinate, which is used to determine the weighting manner of the surrounding pixel points; represents the pixel value at the coordinate of the original image, i.e. the original pixel value participating in the weighted average calculation; represents the filter kernel used in Gaussian filtering, which is a two-dimensional matrix, and the size and weight value distribution determine the filtering effect; An adaptive weight is introduced to optimize the Gaussian filter algorithm: First consider the local variance: compute the variance of the original image within a local region centered at ; and ​ Secondly, adjust the weight: according to the local variance to adjust the weight of the filter kernel ; The final optimized formula, i.e. the filtered pixel value, becomes: ； wherein, is the adjusted weight; In addition, the Gaussian filtering algorithm is optimized in combination with multi-scale filtering: First, a plurality of Gaussian filtering kernels of different scales are used; for each scale of the filtering kernel, the filtered pixel value is calculated according to the original formula; Then weighted fusion is performed: the results of different scale filtering are weighted and fused; let the pixel values filtered by different scales be respectively , wherein represents a scale, and the weight is ; and the optimized pixel value is: ； wherein .

6. The method of claim 1, wherein the method further comprises: In step S2, the model is constructed in a combination of a traditional convolutional neural network CNN and a Vision Transformers convolutional neural network; the traditional convolutional neural network CNN includes convolution operation, pooling operation and full connection layer; The principle of convolution is as follows: ； wherein, is the pixel value of the convolution output, representing the new pixel value obtained after the convolution operation at the coordinate ; is the pixel value of the input image, i.e. the pixel value of the original eye image at the coordinate ; is the convolution kernel, which is a small matrix used for sliding convolution operation on the image, and its size and element value determine the effect of convolution; is the bias, which is a constant term used to adjust the position of the convolution result; is the activation function, which is used to introduce a non-linear factor to increase the expression ability of the model; The purpose of the pooling operation is to reduce the amount of data while retaining important feature information; common pooling methods include maximum pooling and average pooling: Maximum pooling selects the maximum value in a small region as the output; Average pooling calculates the average value of pixel values in a small region as the output; The full connection layer is used to integrate the feature vectors obtained by the previous convolution and pooling operations and map them to the final output dimension; each neuron of the full connection layer is connected to all neurons of the previous layer.

7. The method of claim 6, wherein the method further comprises: determining a contrast sensitivity function of the user based on the captured images. The core of the Vision Transformers convolutional neural network is the self-attention mechanism; it can calculate the correlation between each position and the remaining positions in the image, thereby better capturing global information; The basic formula of self-attention is as follows: ； wherein, are respectively a query vector, a key vector and a value vector obtained by performing linear transformation on input features; is a dimension of the key vector; is a function for normalizing the calculated weights so that their sum is 1; The Vision Transformers convolutional neural network also includes a multi-layer perceptron MLP: after the self-attention mechanism, a multi-layer perceptron is usually connected to further process the features; the MLP is composed of multiple full connection layers and is used to perform nonlinear transformation on the features processed by the self-attention mechanism to increase the expression ability of the model.

8. The method of claim 1, wherein the method further comprises: determining a contrast sensitivity function of the user based on the captured images. In step S2, the model application includes the following steps: A1, mobile terminal device adaptation Download a special mobile application and be guided to collect and take clear eye images in the application interface; A2, eye image shooting and uploading The completed eye images are automatically saved in the local storage of the mobile terminal device; the user selects to upload the images immediately for analysis; A3, vision monitoring result display Once the image is uploaded successfully, the vision monitoring model on the server side will immediately analyze the image; during the analysis process, the model will extract the features of the eye image and compare them with the pre-trained model to determine whether the user's eyes are myopic or normal; the analysis result will be returned to the mobile terminal device within a short time and displayed to the user on the application interface.

9. The method of claim 1, wherein the method further comprises: determining a contrast sensitivity function of the user based on the captured images. In step S3, the model is trained using the cross-validation method, which specifically includes: S3.1, dividing the eye image data set treated in step S1 into subsets of equal size; S3.2, performing Training and validation: in each round, one of the subsets is selected as the validation set, and the rest of the subsets are selected as the training set; the model is trained using the training set, and in the training process, the model learns the relationship between the features of the eye images in the training set and the corresponding myopia or normal eye labels; Training and validation: in each round, one of the subsets is selected as the validation set, and the rest of the subsets are selected as the training set; the model is trained using the training set, and in the training process, the model learns the relationship between the features of the eye images in the training set and the corresponding myopia or normal eye labels; S3.3, after each round of training, the model is verified using the validation set of the current round to evaluate the performance of the model on data that did not participate in training.

10. The method of claim 9, wherein the method further comprises: In step S3, the parameters used to evaluate the classification performance of the model include at least accuracy, precision, recall and F1 score, and the F1 score is used as the final evaluation standard; wherein: Accuracy refers to the proportion of the number of correctly predicted positive or negative examples to the total number of samples, and the calculation formula is: ； Precision is the ratio of the number of samples predicted as positive and actually positive to the number of samples predicted as positive, and the calculation formula is: ； Recall is the ratio of the number of samples predicted as positive and actually positive to the number of samples actually positive, and the calculation formula is: ； wherein, represents the number of samples predicted as positive and actually positive, represents the number of samples predicted as positive and actually negative, represents the number of samples predicted as negative and actually positive, represents the number of samples predicted as negative and actually negative; represents the number of samples predicted as positive; represents the number of samples actually positive; F1 score is an evaluation index that considers precision and recall comprehensively, and the formula is: ； It provides a more comprehensive model performance evaluation to balance the relationship between accuracy and recall.

Images