Vision detection method and system based on infrared thermopile sensor

Abstract

The invention discloses a vision detection method and system based on an infrared thermopile sensor, and belongs to the technical field of image recognition, and the method comprises the steps: displaying a random E character in an opening direction according to a preset rule; acquiring a gesture thermal image sequence of the subject in a non-contact manner by using an infrared thermopile sensor array; the thermal image sequence is processed and recognized through a BP neural network with the initial weight optimized through a genetic algorithm, and the gesture direction is obtained; comparing the gesture direction with the opening direction of the current E character in real time, judging whether to switch the size of the test E character or stop the test based on the self-adaptive logic of the accumulated correct response rate, and finally determining the vision level of the subject. Detection can be completed without contact, convenience and comfort are improved, recognition precision is improved in combination with the optimized neural network, the detection process is scientific, and the method is suitable for various screening scenes.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to a vision detection method and system based on an infrared thermopile sensor. Background Technology

[0002] Currently, there are over 300 million people with myopia, and approximately 450,000 new cases of blindness and 1.35 million cases of low vision occur annually, meaning roughly one person becomes blind and three people develop low vision every minute. China has the highest number of people with myopia in the world, and its incidence rate is second only to Japan. Currently, vision testing still uses visual acuity charts, requiring manual guidance for the test subjects. However, due to limitations in the space and light intensity of the testing environment, traditional visual acuity chart testing methods are inaccurate, affecting the vision correction and protection of myopia patients, and have significant limitations in use. Furthermore, manual testing is inefficient.

[0003] Current vision testing systems mostly fall into two categories: one is based on eye charts. The principle behind eye chart vision testing is that the higher the degree of myopia, the larger the visual field limit of the human eye; conversely, the better the vision, the smaller the visual field limit. The other type uses specialized instruments to measure the refractive power of the eyeball. However, specialized instruments are cumbersome to operate, costly, and difficult to popularize; while traditional eye chart vision testing is inefficient and requires specialized personnel, making it difficult to detect changes in vision promptly. Most new self-testing vision products on the market are based on eye charts and use methods such as voice control, camera image recognition control, and remote control for self-testing. However, voice control is susceptible to environmental interference and has high environmental requirements; camera image recognition is also affected by light intensity; and remote control requires the user to select buttons, which is cumbersome. Therefore, providing a vision testing method and system that allows for self-testing at home with high accuracy is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the present invention provides a vision detection method and system based on an infrared thermopile sensor. By displaying the character "E" in a random direction and using a non-contact infrared sensor array to collect and identify the user's gesture direction, vision comparison and judgment are automatically completed. This solves the problems of traditional detection methods relying on manual labor and having contact risks or operational errors, and realizes the automation, objectivity and efficiency of vision detection.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: On one hand, the present invention provides a vision detection method based on an infrared thermopile sensor, comprising: Step 1: Display an E character with a random opening direction according to preset rules; Step 2: Acquire a sequence of thermal images of the subject's response gestures to the current E character using an infrared thermopile sensor array; Step 3: Preprocess the acquired thermal image sequence and output the gesture recognition result through the trained BP neural network model; Step 4: Compare the gesture recognition result with the opening direction of the current E character, and determine the subject's final visual acuity value based on the comparison result; Step 5: Store and display the subject's final visual acuity value in real time.

[0006] Preferably, step 4 includes: Step 4.1: Compare the recognized gesture direction with the opening direction of the current E character; Step 4.2: If the comparison result shows that the directions are consistent, record it as a normal response and proceed to step 4.4; Step 4.3: If the comparison results are inconsistent or invalid, the row is determined to be unrecognizable, and step 4.5 is executed; Step 4.4: Determine the number of correct responses under the current size. If the number of consecutive correct responses for the E character of the same size is greater than or equal to the preset number, randomly display the E character of the next size and repeat steps 2-4; if the number of consecutive correct responses for the E character of the same size is less than the preset number, proceed to step 4.5. Step 4.5: Determine the percentage of normal responses in the total number of responses. If the percentage is greater than the preset value, randomly display the next size of the E character and repeat steps 2-4. If the percentage is less than or equal to the preset value, stop the detection and take the visual acuity level corresponding to the previous size of the E character as the final visual acuity value.

[0007] Preferably, the specific structure of the BP neural network is as follows: The BP neural network adopts a 4-layer fully connected feedforward network structure, including 1 input layer, 2 hidden layers and 1 output layer, with each layer connected in series. The number of neurons in the input layer is the same as the total number of pixels in the preprocessed single-frame gesture thermal image. It is used to receive the standardized thermal image pixel feature data. No activation function is set in the input layer, and the pixel feature data is directly transmitted to the first hidden layer. The number of neurons in the first hidden layer is set to 512, and the ReLU activation function is used to extract shallow nonlinear features of the gesture thermal image. The second hidden layer has 64 neurons and uses the ReLU activation function to further extract deep abstract features from the gesture thermal image. The output layer has 4 neurons, which correspond to the four valid gestures of the character E: up, down, left, and right. The Softmax activation function is used to output the probability value of each state. The state with the highest probability is the final gesture recognition result. The input layer is fully connected to the first hidden layer, the first hidden layer is connected to the second hidden layer, and the second hidden layer is connected to the output layer. The neurons in adjacent layers are mapped to each other through trainable weight parameters. The weights are initialized using a genetic algorithm, and the bias term is initialized to 0.

[0008] Preferably, the BP neural network is trained using training set samples, and the neural network weights are initialized using a genetic algorithm, specifically including: The population size is set, and each individual is encoded as a real vector containing all weights and bias parameters from the input layer to the first hidden layer, from the first hidden layer to the second hidden layer, and from the second hidden layer to the output layer. The fitness function is designed as a weighted sum of the classification accuracy and loss function value of the network constructed by the individual on the validation set. After M generations of iteration, the parent generation is selected by roulette wheel selection strategy in each generation, and the offspring generation is generated through single-point crossover and mutation operations. Finally, the individual with the highest fitness is selected as the initial weight of the network. The weight parameters optimized by the genetic algorithm are assigned to the BP neural network, and the Adam optimization algorithm is used for supervised training. The cross-entropy loss function is used as the loss function, and an L2 regularization term is added to prevent overfitting. An early stopping strategy is adopted during training. Training is stopped when the validation set loss no longer decreases for N consecutive rounds or the validation set accuracy reaches or exceeds the preset accuracy. The optimal model parameters are saved to obtain a trained BP neural network model for gesture recognition.

[0009] Preferably, the infrared thermopile sensor array is 4 4 or 5 One of the five.

[0010] Preferably, step 2 specifically includes: Step 2.1: Upon receiving the E character display prompt, the gesture detection mode is activated, and the thermal radiation data of the subject's palm area is collected in real time through the infrared thermopile sensor array to form the original thermal image frame sequence; Step 2.2: Using the background subtraction method, the real-time acquired raw thermal image frame sequence is subjected to pixel-by-pixel difference calculation with the reference frame to obtain the foreground gesture thermal image frame; Step 2.3: Perform temperature threshold filtering on the foreground gesture thermal image frames to retain valid pixels; Step 2.4: Based on the temperature gradient change of effective pixels, detect the palm area. When an effective area matching the temperature characteristics of the gesture is detected for X consecutive frames and the area of ​​the area remains unchanged, it is determined to be the effective gesture start frame, and the recording of the thermal image sequence begins. Step 2.5: Starting from the effective gesture start frame, continuously acquire preset frame thermal image data to form a complete gesture thermal image sequence; Step 2.6: Timestamp the collected gesture thermal image sequences and package them into data packets according to the collection order.

[0011] Preferably, step 2 further includes: real-time monitoring of the distance between the subject's palm and the infrared thermopile sensor array; when the distance exceeds a preset fluctuation range, pausing the acquisition and guiding the subject to adjust their position.

[0012] On the other hand, the present invention provides a vision detection system based on an infrared thermopile sensor, comprising: The display module is used to display E characters with random opening directions according to preset rules; An infrared thermopile sensor array is positioned in front of the display module to acquire a sequence of thermal images of the subject's response gestures in response to the currently displayed character "E". The gesture recognition module is communicatively connected to the infrared thermopile sensor array module. It is used to preprocess the acquired thermal image sequence and output the gesture recognition result through the built-in trained BP neural network model. The judgment module is connected to the display module and the gesture recognition module respectively. It is used to compare the gesture recognition result with the opening direction of the current E character and determine the subject's final visual acuity value according to the preset comparison rules. The result output module, connected to the judgment module, is used to store and display the final visual acuity value in real time.

[0013] As can be seen from the above technical solution, compared with the prior art, this invention discloses a vision detection method and system based on an infrared thermopile sensor. By integrating an infrared sensor array and a trained BP neural network, it can automatically collect and accurately identify the user's gesture direction, replacing the manual interpretation and response step in traditional detection, effectively eliminating errors caused by the subjective experience of the testing personnel, and making the detection results more accurate and reliable. Using non-contact infrared gesture recognition, users do not need to touch any physical buttons or devices, avoiding the risk of cross-infection and solving the operational difficulties of children, people with mobility impairments, or special groups. The detection process is natural and hygienic, and the system has real-time guidance and distance monitoring functions, further optimizing the human-computer interaction experience. Furthermore, this invention uses a low-cost infrared thermopile sensor array, combined with an optimized neural network model, reducing the overall hardware cost and computational overhead while ensuring high performance. This makes the system easy to deploy in various scenarios such as community clinics, schools, and homes, which is conducive to promoting the popularization of vision screening. Furthermore, based on multi-round, multi-size statistical comparison and intelligent judgment, the testing progress can be dynamically adjusted to accurately determine the final vision value. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the process provided by the present invention.

[0016] Figure 2 This is a diagram of the BP neural network structure.

[0017] Figure 3 A diagram of a BP neural network structure optimized for a genetic algorithm.

[0018] Figure 4 This is the hardware circuit diagram of an infrared thermopile array.

[0019] Figure 5 This is a picture of the actual Android app.

[0020] Figure 6 This is a structural schematic diagram provided for the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] This invention discloses a vision detection method based on an infrared thermopile sensor, such as... Figure 1 As shown, it includes: Step 1: Display E characters with random opening directions according to preset rules; specifically, control the LCD screen to display E characters sequentially according to the levels of the standard logarithmic visual acuity chart, starting with the E character of visual acuity value 0.1, and sequentially displaying E characters of visual acuity levels such as 0.2, 0.3, ..., 1.0, 1.2, 1.5, 2.0, etc.; display 5-10 E characters under each visual acuity level, corresponding to the four opening directions of up, down, left, and right, respectively, and the display time of each E character is 3-5 seconds to ensure that the examinee can observe clearly.

[0023] Step 2: Acquire a sequence of thermal images of the subject's response gestures to the current E character using an infrared thermopile sensor array; Step 3: Preprocess the acquired thermal image sequence and output the gesture recognition result through the trained BP neural network model; the preprocessing includes using Arduino to perform mean filtering and normalization to the (0,1) interval on the acquired gesture images; The formula for the normalized exponential function is:

[0024] In the formula, z j Let j represent the j-th element. This function compresses a K-dimensional vector containing arbitrary real numbers into another K-dimensional real vector, such that each element is in the range (0,1).

[0025] Step 4: Compare the gesture recognition result with the opening direction of the current E character, and determine the subject's final visual acuity value based on the comparison result; Step 5: Store and display the subject's final visual acuity value in real time. The obtained visual acuity value is uploaded to Tencent Cloud server via TCP / IP communication protocol. The visual acuity value is read and displayed via an Android mobile app, showing both current and historical values. (See image for a physical screenshot of the mobile app.) Figure 5 As shown.

[0026] Furthermore, step 4 includes: Step 4.1: Compare the recognized gesture direction with the opening direction of the current E character; Step 4.2: If the comparison result shows that the directions are consistent, record it as a normal response and proceed to step 4.4; Step 4.3: If the comparison results are inconsistent or invalid, the row is determined to be unrecognizable, and step 4.5 is executed; Step 4.4: Determine the number of correct responses under the current size. If the number of consecutive correct responses for the E character of the same size is greater than or equal to the preset number, randomly display the E character of the next size and repeat steps 2-4; if the number of consecutive correct responses for the E character of the same size is less than the preset number, proceed to step 4.5. Step 4.5: Determine the percentage of normal responses in the total number of responses. If the percentage is greater than the preset value, randomly display the next size of the E character and repeat steps 2-4. If the percentage is less than or equal to the preset value, stop the detection and take the visual acuity level corresponding to the previous size of the E character as the final visual acuity value.

[0027] Specifically, such as Figure 2 As shown, the specific structure of the BP neural network is as follows: The BP neural network adopts a 4-layer fully connected feedforward network structure, including 1 input layer, 2 hidden layers and 1 output layer, with each layer connected in series. The number of neurons in the input layer is the same as the total number of pixels in the preprocessed single-frame gesture thermal image. It is used to receive the standardized thermal image pixel feature data. No activation function is set in the input layer, and the pixel feature data is directly transmitted to the first hidden layer. The number of neurons in the first hidden layer is set to 512, and the ReLU activation function is used to extract shallow nonlinear features of the gesture thermal image. The second hidden layer has 64 neurons and uses the ReLU activation function to further extract deep abstract features from the gesture thermal image. The output layer has 4 neurons, which correspond to the four valid gestures of the character E: up, down, left, and right. The Softmax activation function is used to output the probability value of each state. The state with the highest probability is the final gesture recognition result. The input layer is fully connected to the first hidden layer, the first hidden layer is connected to the second hidden layer, and the second hidden layer is connected to the output layer. The neurons in adjacent layers are mapped to each other through trainable weight parameters. The weights are initialized using a genetic algorithm, and the bias term is initialized to 0.

[0028] The training and test sets were divided in a 7:3 ratio, and data augmentation was performed using methods such as image translation and flipping to avoid overfitting in the neural network. Figure 3 As shown, the BP neural network is trained using training set samples, and the neural network weights are initialized using a genetic algorithm, specifically including: The population size is set, and each individual is encoded as a real vector containing all weights and bias parameters from the input layer to the first hidden layer, from the first hidden layer to the second hidden layer, and from the second hidden layer to the output layer. The fitness function is designed as a weighted sum of the classification accuracy and loss function value of the network constructed by the individual on the validation set. After M generations of iteration, the parent generation is selected by roulette wheel selection strategy in each generation, and the offspring generation is generated through single-point crossover and mutation operations. Finally, the individual with the highest fitness is selected as the initial weight of the network. The weight parameters optimized by the genetic algorithm are assigned to the BP neural network, and the Adam optimization algorithm is used for supervised training. The cross-entropy loss function is used as the loss function, and an L2 regularization term is added to prevent overfitting. An early stopping strategy is adopted during training. Training is stopped when the validation set loss no longer decreases for N consecutive rounds or the validation set accuracy reaches or exceeds the preset accuracy. The optimal model parameters are saved to obtain a trained BP neural network model for gesture recognition.

[0029] like Figure 4 As shown, the infrared thermopile sensor array is 4 4 or 5 One type of 5, wherein the infrared thermopile is one of XGZT263, XGZG360, or XGZT264. The sensor output signal processing system includes an AD analog-to-digital converter, a memory, and a microprocessor. The amplifier circuit consists of an LM358N non-inverting amplifier and resistors with resistances of 100KΩ and 100Ω respectively.

[0030] In another embodiment, step 2 specifically includes: Step 2.1: Upon receiving the E character display prompt, the gesture detection mode is activated, and the thermal radiation data of the subject's palm area is collected in real time through the infrared thermopile sensor array to form the original thermal image frame sequence; Step 2.2: Using the background subtraction method, the real-time acquired raw thermal image frame sequence is subjected to pixel-by-pixel difference calculation with the reference frame to obtain the foreground gesture thermal image frame; Step 2.3: Perform temperature threshold filtering on the foreground gesture thermal image frames to retain valid pixels; Step 2.4: Based on the temperature gradient change of effective pixels, detect the palm area. When an effective area matching the temperature characteristics of the gesture is detected for X consecutive frames and the area of ​​the area remains unchanged, it is determined to be the effective gesture start frame, and the recording of the thermal image sequence begins. Step 2.5: Starting from the effective gesture start frame, continuously acquire preset frame thermal image data to form a complete gesture thermal image sequence; Step 2.6: Timestamp the collected gesture thermal image sequences and package them into data packets according to the collection order.

[0031] In another embodiment, step 2 further includes: real-time monitoring of the distance between the subject's palm and the infrared thermopile sensor array; when the distance exceeds a preset fluctuation range, pausing the acquisition and guiding the subject to adjust their position.

[0032] This invention is low-cost and produces highly accurate results. The neural network achieved a 96.7% accuracy rate in classifying the four gestures on the test set, and the classification results also showed good robustness in actual testing. Furthermore, the interaction between the Android app and the server enables data sharing between doctors and users, allowing doctors to promptly monitor changes in the user's vision.

[0033] On the other hand, the present invention provides a vision detection system based on an infrared thermopile sensor, such as... Figure 6 As shown, it includes: The display module is used to display E characters with random opening directions according to preset rules; An infrared thermopile sensor array is positioned in front of the display module to acquire a sequence of thermal images of the subject's response gestures in response to the currently displayed character "E". The gesture recognition module is communicatively connected to the infrared thermopile sensor array module. It is used to preprocess the acquired thermal image sequence and output the gesture recognition result through the built-in trained BP neural network model. The judgment module is connected to the display module and the gesture recognition module respectively. It is used to compare the gesture recognition result with the opening direction of the current E character and determine the subject's final visual acuity value according to the preset comparison rules. The result output module, connected to the judgment module, is used to store and display the final visual acuity value in real time.

[0034] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0035] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A vision detection method based on an infrared thermopile sensor, characterized by, include: Step 1: Display an E character with a random opening direction according to preset rules; Step 2: Acquire a sequence of thermal images of the subject's response gestures to the current E character using an infrared thermopile sensor array; Step 3: Preprocess the acquired thermal image sequence and output the gesture recognition result through the trained BP neural network model; Step 4: Compare the gesture recognition result with the opening direction of the current E character, and determine the subject's final visual acuity value based on the comparison result; Step 5: Store and display the subject's final visual acuity value in real time.

2. The vision detection method based on an infrared thermopile sensor according to claim 1, wherein, Step 4 includes: Step 4.1: Compare the recognized gesture direction with the opening direction of the current E character; Step 4.2: If the comparison result shows that the directions are consistent, record it as a normal response and proceed to step 4.4; Step 4.3: If the comparison results are inconsistent or invalid, the row is determined to be unrecognizable, and step 4.5 is executed; Step 4.4: Determine the number of correct responses under the current size. If the number of consecutive correct responses for the E character of the same size is greater than or equal to the preset number, randomly display the E character of the next size and repeat steps 2-4; if the number of consecutive correct responses for the E character of the same size is less than the preset number, proceed to step 4.

5. Step 4.5: Determine the percentage of normal responses in the total number of responses. If the percentage is greater than the preset value, randomly display the next size of the E character and repeat steps 2-4. If the percentage is less than or equal to the preset value, stop the detection and take the visual acuity level corresponding to the previous size of the E character as the final visual acuity value. 3.The vision detection method based on an infrared thermopile sensor according to claim 1, wherein, The specific structure of the BP neural network is as follows: The BP neural network adopts a 4-layer fully connected feedforward network structure, including 1 input layer, 2 hidden layers and 1 output layer, with each layer connected in series. The number of neurons in the input layer is the same as the total number of pixels in the preprocessed single-frame gesture thermal image. It is used to receive the standardized thermal image pixel feature data. No activation function is set in the input layer, and the pixel feature data is directly transmitted to the first hidden layer. The number of neurons in the first hidden layer is set to 512, and the ReLU activation function is used to extract shallow nonlinear features of the gesture thermal image. The second hidden layer has 64 neurons and uses the ReLU activation function to further extract deep abstract features from the gesture thermal image. The output layer has 4 neurons, which correspond to the four valid gestures of the character E: up, down, left, and right. The Softmax activation function is used to output the probability value of each state. The state with the highest probability is the final gesture recognition result. The input layer is fully connected to the first hidden layer, the first hidden layer is connected to the second hidden layer, and the second hidden layer is connected to the output layer. The neurons in adjacent layers are mapped to each other through trainable weight parameters. The weights are initialized using a genetic algorithm, and the bias term is initialized to 0.

4. The vision detection method based on an infrared thermopile sensor according to claim 1, wherein, The BP neural network is trained using training set samples, and the neural network weights are initialized using a genetic algorithm, specifically including: The population size is set, and each individual is encoded as a real vector containing all weights and bias parameters from the input layer to the first hidden layer, from the first hidden layer to the second hidden layer, and from the second hidden layer to the output layer. The fitness function is designed as a weighted sum of the classification accuracy and loss function value of the network constructed by the individual on the validation set. After M generations of iteration, the parent generation is selected by roulette wheel selection strategy in each generation, and the offspring generation is generated through single-point crossover and mutation operations. Finally, the individual with the highest fitness is selected as the initial weight of the network. The weight parameters optimized by the genetic algorithm are assigned to the BP neural network, and the Adam optimization algorithm is used for supervised training. The cross-entropy loss function is used as the loss function, and an L2 regularization term is added to prevent overfitting. An early stopping strategy is adopted during training. Training is stopped when the validation set loss no longer decreases for N consecutive rounds or the validation set accuracy reaches or exceeds the preset accuracy. The optimal model parameters are saved to obtain a trained BP neural network model for gesture recognition.

5. The vision detection method based on an infrared thermopile sensor according to claim 1, wherein, The infrared thermopile sensor array is 4 4 or 5 5.

6. The vision detection method based on an infrared thermopile sensor according to claim 1, wherein, Step 2 specifically includes: Step 2.1: Upon receiving the E character display prompt, the gesture detection mode is activated, and the thermal radiation data of the subject's palm area is collected in real time through the infrared thermopile sensor array to form the original thermal image frame sequence; Step 2.2: Using the background subtraction method, the real-time acquired raw thermal image frame sequence is subjected to pixel-by-pixel difference calculation with the reference frame to obtain the foreground gesture thermal image frame; Step 2.3: Perform temperature threshold filtering on the foreground gesture thermal image frames to retain valid pixels; Step 2.4: Based on the temperature gradient change of effective pixels, detect the palm area. When an effective area matching the temperature characteristics of the gesture is detected for X consecutive frames and the area of ​​the area remains unchanged, it is determined to be the effective gesture start frame, and the recording of the thermal image sequence begins. Step 2.5: Starting from the effective gesture start frame, continuously acquire preset frame thermal image data to form a complete gesture thermal image sequence; Step 2.6: Timestamp the collected gesture thermal image sequences and package them into data packets according to the collection order.

7. The vision detection method based on an infrared thermopile sensor according to claim 6, wherein, Step 2 also includes: real-time monitoring of the distance between the subject's palm and the infrared thermopile sensor array; when the distance exceeds the preset fluctuation range, the data acquisition is paused and the subject is guided to adjust their position.

8. A vision detection system based on infrared thermopile sensors, characterized in that, include: The display module is used to display E characters with random opening directions according to preset rules; An infrared thermopile sensor array is positioned in front of the display module to acquire a sequence of thermal images of the subject's response gestures in response to the currently displayed character "E". The gesture recognition module is communicatively connected to the infrared thermopile sensor array module. It is used to preprocess the acquired thermal image sequence and output the gesture recognition result through the built-in trained BP neural network model. The judgment module is connected to the display module and the gesture recognition module respectively. It is used to compare the gesture recognition result with the opening direction of the current E character and determine the subject's final visual acuity value according to the preset comparison rules. The result output module, connected to the judgment module, is used to store and display the final visual acuity value in real time.

Images