Myopia prevention and control and coordinate correction device based on multi-mode perception

Abstract

The invention discloses a myopia prevention and control and coordinate correction device based on multi-modal perception, and the device is provided with a multi-modal perception unit, an algorithm processing unit, an intelligent feedback unit and a far-end cooperation platform, and the multi-modal perception unit comprises an image recognition module, a motion sensor array and an ambient light sensor. The image recognition module captures a face image of a user, the face image is recognized and analyzed through a convolutional neural network to obtain eye information of the user, the motion sensor array detects head motion information and sitting posture information of the user, and the ambient light sensor monitors illumination intensity and color temperature information in real time and evaluates the visual atmosphere. A machine learning model is used for recognizing and analyzing micro expressions of a facial image, the sitting posture and the eyestrain state of a user can be precisely recognized, the bad sitting posture of the user is intervened in time, algorithm-driven active intervention is introduced, a reminding strategy is dynamically adjusted according to the state of the user, convenience is provided for personalized eyestrain and sitting posture of the user, and the user experience is improved. And the use experience of the product is also improved.

Description

Technical Field

[0001] This invention belongs to the field of wearable device technology, and particularly relates to a myopia prevention and coordinate correction device based on multimodal perception. Background Technology

[0002] In recent years, myopia among children and adolescents has become increasingly serious, posing a global public health challenge. Existing prevention and control measures, such as wearing glasses and vision training, lack real-time and personalized intervention capabilities. Meanwhile, poor posture and eye habits further exacerbate myopia development, and traditional devices struggle to comprehensively monitor these factors. Current smart glasses primarily focus on basic health monitoring, such as heart rate and step count, but lack comprehensive perception across multiple dimensions, including the eye-use environment, posture, and attention. Existing technologies have significant shortcomings in image recognition, voice interaction, and personalized algorithms, making precise intervention difficult. Therefore, there is an urgent need to provide a myopia prevention and coordinate correction device based on multimodal perception to address these technical challenges. Summary of the Invention

[0003] In view of this, the present invention provides a myopia prevention and coordinate correction device based on multimodal perception, which can comprehensively perceive sitting posture, ambient light, facial expression and attention through multi-sensor fusion, and use machine learning models to provide personalized eye use and sitting posture correction strategies for different users, thereby improving the flexibility, convenience and working stability of the device. The specific technical solution adopted is as follows.

[0004] This invention provides a myopia prevention and coordinate correction device based on multimodal perception, comprising: The multimodal perception unit includes an image recognition module, a motion sensor array, and an ambient light sensor. The image recognition module is used to capture the user's facial image in real time and uses a convolutional neural network to recognize and analyze the facial image to obtain the user's eye information. The motion sensor array is used to detect the user's head movement information and sitting posture information. The ambient light sensor is used to monitor the light intensity and color temperature information corresponding to the facial image in real time and evaluate the user's visual atmosphere based on the light intensity and color temperature information. An algorithm processing unit, connected to the multimodal perception unit, is used to run a lightweight deep learning model corresponding to the convolutional neural network and to perform fusion analysis on multi-source data to construct user behavior profile information. The multi-source data includes facial images, head movement information and sitting posture information, light intensity and color temperature information. The intelligent feedback unit includes a multi-mode reminder module and an atmosphere adaptation module. The multi-mode reminder module is used to receive the output of the algorithm processing unit and trigger interactive feedback information. The atmosphere adaptation module adjusts the user's visual environment according to the light intensity. The remote collaboration platform is used to receive and store the multi-source data, user behavior profile information and interactive feedback information, and generate corresponding personalized sitting posture rules.

[0005] As a preferred embodiment of the above technical solution, the execution process of the image recognition module includes: Multiple facial images of the user are acquired, and the facial images are preprocessed to obtain a facial image. The preprocessing includes face alignment and size normalization. An optical flow estimation algorithm is used to analyze the pixel intensity of adjacent image frames of a facial image, and a two-dimensional displacement vector is used to represent dynamic changes to determine the motion vector of each pixel in each frame. The specific analysis process is as follows: In the preset expression sample frame at time t The pixel intensity at that location is After time The distance moved later becomes Its brightness value becomes Based on the principle of constant brightness between consecutive frames, the expression is: (1) Expanding and simplifying equation (1) using Taylor series, the corresponding optical flow equation is: (2) Remove general form Then substitute formula (1) into formula (2) and divide by . Formula (3) is obtained as follows: (3) in, The derivative is used to obtain the estimated vectors of horizontal and vertical optical flow velocity of micro-expressions by solving formula (3).

[0006] As a preferred embodiment of the above technical solution, a preset and The coordinates of the key points at the left and right inner corners of the eye are given. Using the midpoint of these two key points as the rotation center, the face alignment calculation process is as follows: (4) in, Horizontal angle of the face Rotated coordinates The facial region is accurately cropped from the original frame, and the facial region that best fits the face is obtained based on the main points around the multiple key points of the cropped area.

[0007] As a preferred embodiment of the above technical solution, a convolutional neural network is used to identify and analyze the facial image to obtain the user's eye information, including: The input to the pre-defined convolutional neural network is... Three-channel optical flow characteristic map The output will be labeled as positive, negative, and surprised. Feed in 24 sizes A regular convolution kernel with a size and stride of 2 is obtained. The shallowest feature F, whose size is ; Max pooling downsampling is performed on the shallowest feature F to reduce the feature dimensionality. The pooling downsampling operation is performed on the... Output of each pooling kernel for: (5) in, and These represent multiplicative factors and additive factors, respectively. This indicates a downsampling operation. Max pooling, on the other hand, uses a sliding window to calculate the maximum value in a specific region of the feature map; the corresponding expression is: (6) in, express medium size The area and The values ​​are 1, 2, and 3; The feature maps are reduced in dimensionality using global average pooling, and classified using a Softmax layer to map the information of each channel of the convolutional neural network.

[0008] As a preferred embodiment of the above technical solution, running the lightweight deep learning model corresponding to the convolutional neural network includes: A dual-attention module is used to enhance the network's ability to identify and analyze multi-source data, with an input size of... Feature mapping matrix Where C, H and W represent the number of channels, height and width of the feature mapping matrix, respectively, and the dual attention module includes the channel attention module CAM and the spatial attention module SAM; Output one-dimensional channel attention mapping matrix sequentially and two-dimensional attention mapping matrix Their sizes are respectively and The expression for the entire derivation process is as follows: (7) (8) in, express and as well as and Multiplying elements at the same position, For output features.

[0009] As a preferred embodiment of the above technical solution, in the channel attention module, it is necessary to reduce the spatial dimension by 1 to generate a one-dimensional channel attention mapping matrix, including: For input features Simultaneously perform max pooling and average pooling operations to obtain different spatial feature descriptors. and ; The attention feature vectors of the two channels and The mixture is fused by addition. The sigmoid activation function is used to normalize the weights, and the attention feature matrix is ​​output. Broadcast it along the spatial dimensions to The specific calculation process is as follows: (9) in, This represents the normalized weights of the activation function. This indicates that a one-dimensional convolution kernel of length k is used for convolution, where the size of k is represented as: (10) Where C represents the number of channels in the feature map. It represents the odd number that is closest to it.

[0010] As a preferred embodiment of the above technical solution, in the spatial attention module, it is necessary to reduce the channel dimension by 1 to generate a two-dimensional spatial attention mapping matrix, including: Input features in the spatial dimension Simultaneously perform max pooling and mean pooling operations to obtain different channel pooling feature descriptors. and ; Two feature descriptors are encoded using convolutional layers, and feature information is extracted in different spatial dimensions using convolutional kernels of different sizes. and Two small convolutional kernels of different sizes form two different spatial attention mapping relationships. and This allows the spatial attention module to analyze the features of different regions; the corresponding expression is: (11) (12) in, and They respectively represent the use and Convolution is performed using convolution kernels. and These represent mean pooling and max pooling, respectively. Attention maps are concatenated using average channel pooling. and Spatial attention features are obtained through the Sigmoid activation function. The corresponding expression is: (13) Formula (11) is the derived mapping relationship of the spatial attention module.

[0011] As a preferred embodiment of the above technical solution, the motion sensor array includes an accelerometer and a gyroscope, and the gyroscope and the accelerometer work together to detect the head tilt angle, head-down state or head-tilt posture corresponding to the head motion information.

[0012] As a preferred embodiment of the above technical solution, when the ambient light intensity is insufficient, the system can fine-tune the lens or remind the user to adjust their sitting position to optimize the visual environment.

[0013] This invention provides a myopia prevention and coordinate correction device based on multimodal perception. It comprises a multimodal perception unit, an algorithm processing unit, an intelligent feedback unit, and a remote collaborative platform. The multimodal perception unit includes an image recognition module, a motion sensor array, and an ambient light sensor. The image recognition module captures the user's facial image in real time and uses a convolutional neural network to analyze the facial image to obtain the user's eye information. The motion sensor array detects the user's head movement and posture. The ambient light sensor monitors light intensity and color temperature in real time and assesses the user's visual atmosphere. A machine learning model is used to identify and analyze micro-expressions in the facial image, accurately identifying the user's posture and eye fatigue state, and providing timely intervention for poor posture. The introduction of algorithm-driven proactive intervention and dynamic adjustment of reminder strategies based on the user's state also provides personalized eye and posture guidance, improving the user experience to some extent. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 The structural block diagram of the myopia prevention and coordinate correction device based on multimodal perception provided by the present invention; Figure 2 The flowchart illustrates the usage method of the myopia prevention and coordinate correction device based on multimodal perception provided by this invention. Detailed Implementation

[0016] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0017] This invention provides a myopia prevention and coordinate correction device based on multimodal perception, comprising: The multimodal perception unit includes an image recognition module, a motion sensor array, and an ambient light sensor. The image recognition module is used to capture the user's facial image in real time and uses a convolutional neural network to recognize and analyze the facial image to obtain the user's eye information. The motion sensor array is used to detect the user's head movement information and sitting posture information. The ambient light sensor is used to monitor the light intensity and color temperature information corresponding to the facial image in real time and evaluate the user's visual atmosphere based on the light intensity and color temperature information. An algorithm processing unit, connected to the multimodal perception unit, is used to run a lightweight deep learning model corresponding to the convolutional neural network and to perform fusion analysis on multi-source data to construct user behavior profile information. The multi-source data includes facial images, head movement information and sitting posture information, light intensity and color temperature information. The intelligent feedback unit includes a multi-mode reminder module and an atmosphere adaptation module. The multi-mode reminder module is used to receive the output of the algorithm processing unit and trigger interactive feedback information. The atmosphere adaptation module adjusts the user's visual environment according to the light intensity. The remote collaboration platform is used to receive and store the multi-source data, user behavior profile information and interactive feedback information, and generate corresponding personalized sitting posture rules.

[0018] In this embodiment, the execution process of the image recognition module includes: acquiring multiple face images of the user, and performing image preprocessing on the face images to obtain a facial image, wherein the image preprocessing includes face alignment and size normalization; using an optical flow estimation algorithm to analyze the pixel intensity of adjacent image frames of the facial image, and using a two-dimensional displacement vector to represent dynamic changes to determine the motion vector of each frame pixel, the specific analysis process is as follows: In the preset expression sample frame at time t The pixel intensity at that location is After time The distance moved later becomes Its brightness value becomes Based on the principle of constant brightness between consecutive frames, the expression is: (1) Expanding and simplifying equation (1) using Taylor series, the corresponding optical flow equation is: (2) Remove general form Then substitute formula (1) into formula (2) and divide by . Formula (3) is obtained as follows: (3) in, The derivative is used to obtain the estimated vectors of horizontal and vertical optical flow velocity of micro-expressions by solving formula (3).

[0019] The myopia prevention and coordinate correction device based on multimodal perception provided by this invention, taking a smart AI mirror as an example, includes a motion sensor array comprising an accelerometer and a gyroscope. The gyroscope and accelerometer work together to detect head tilt angles, head-down postures, or head-tilts corresponding to head movement information. When the ambient light intensity is insufficient, the system fine-tunes the lens or reminds the user to adjust their sitting position to optimize the visual environment. In other words, in the multimodal perception unit, the image recognition module integrates a miniature camera to capture the user's facial images in real time. It analyzes facial features, blink frequency, pupil diameter, etc., through a convolutional neural network (CNN) to determine attention levels and eye fatigue. The motion sensor array, combined with the accelerometer and gyroscope, detects head tilt angles and body sitting angles to identify poor postures such as head-down and head-tilts. The ambient light sensor monitors light intensity and color temperature in real time to assess whether the visual atmosphere is suitable.

[0020] The algorithm matching and processing engine (algorithm processing unit) employs an embedded AI chip to run a lightweight deep learning model, fusing and analyzing multi-source data (images, motion, light signals). It builds user behavior profiles through personalized learning algorithms, dynamically adjusting monitoring thresholds and feedback strategies. The intelligent feedback system (intelligent feedback unit) includes multi-mode reminders and ambient adaptation. Multi-mode reminders trigger vibration motors, LED indicators, or voice prompts (such as "Please look up and into the distance") based on algorithm output, enabling natural interaction. Ambient adaptation optimizes the visual environment by fine-tuning lenses or suggesting the user adjust their position when ambient light is insufficient. The remote collaboration platform (parental collaboration platform) synchronizes data to the parent's mobile app via Bluetooth or Wi-Fi, providing detailed reports including the number of times poor posture was observed, attention curves, and outdoor time, and supports remotely setting personalized rules.

[0021] It should be noted that the default setting... and The coordinates of the key points at the left and right inner corners of the eye are given. Using the midpoint of these two key points as the rotation center, the face alignment calculation process is as follows: (4) in, Horizontal angle of the face Rotated coordinates The facial region is accurately cropped from the original frame, and the facial region that best fits the face is obtained based on the main points around the multiple key points of the cropped area.

[0022] It should be understood that by setting up a multimodal perception unit, an algorithm processing unit, an intelligent feedback unit, and a remote collaboration platform, the multimodal perception unit includes an image recognition module, a motion sensor array, and an ambient light sensor. The image recognition module captures the user's facial images in real time and uses a convolutional neural network to identify and analyze the facial images to obtain the user's eye information. The motion sensor array detects the user's head movement and posture information. The ambient light sensor monitors the light intensity and color temperature information in real time and assesses the user's visual atmosphere. The machine learning model is used to identify and analyze the micro-expressions in the facial images, which can accurately identify the user's posture and eye fatigue state, and intervene in the user's poor posture in a timely manner. The introduction of algorithm-driven proactive intervention and dynamic adjustment of reminder strategies according to the user's status also provides users with personalized eye and posture convenience, and to a certain extent improves the user experience of the product.

[0023] Optionally, a convolutional neural network is used to perform recognition and analysis on the facial image to obtain the user's eye information, including: The input to the pre-defined convolutional neural network is... Three-channel optical flow characteristic map The output will be labeled as positive, negative, and surprised. Feed in 24 sizes A regular convolution kernel with a size and stride of 2 is obtained. The shallowest feature F, whose size is ; Max pooling downsampling is performed on the shallowest feature F to reduce the feature dimensionality. The pooling downsampling operation is performed on the... Output of each pooling kernel for: (5) in, and These represent multiplicative factors and additive factors, respectively. This indicates a downsampling operation. Max pooling, on the other hand, uses a sliding window to calculate the maximum value in a specific region of the feature map; the corresponding expression is: (6) in, express medium size The area and The values ​​are 1, 2, and 3; The feature maps are reduced in dimensionality using global average pooling, and classified using a Softmax layer to map the information of each channel of the convolutional neural network.

[0024] In this embodiment, running the lightweight deep learning model corresponding to the convolutional neural network includes: using a dual attention module to enhance network recognition and analysis of multi-source data, and inputting data of size [value missing]. Feature mapping matrix Where C, H and W represent the number of channels, height and width of the feature mapping matrix, respectively, and the dual attention module includes the channel attention module CAM and the spatial attention module SAM; Output one-dimensional channel attention mapping matrix sequentially and two-dimensional attention mapping matrix Their sizes are respectively and The expression for the entire derivation process is as follows: (7) (8) in, express and as well as and Multiplying elements at the same position, For output features.

[0025] It should be noted that in the channel attention module, in order to effectively determine the attention of the model and extract the main information of the target object, the spatial dimension needs to be reduced by 1 to generate a one-dimensional channel attention mapping matrix, including: input features Simultaneously perform max pooling and average pooling operations to obtain different spatial feature descriptors. and ; The attention feature vectors of the two channels and The mixture is fused by addition. The sigmoid activation function is used to normalize the weights, and the attention feature matrix is ​​output. Broadcast it along the spatial dimensions to The specific calculation process is as follows: (9) in, This represents the normalized weights of the activation function. This indicates that a one-dimensional convolution kernel of length k is used for convolution, where the size of k is represented as: (10) Where C represents the number of channels in the feature map. It represents the odd number that is closest to it.

[0026] Specifically, in the spatial attention module, the channel dimension needs to be reduced by 1 to generate a two-dimensional spatial attention mapping matrix, including: Input features in the spatial dimension Simultaneously perform max pooling and mean pooling operations to obtain different channel pooling feature descriptors. and This can more effectively highlight information regions. Then, the two feature descriptors are connected by channel overlay, and then they are encoded by convolutional layers to emphasize or suppress information at specific locations.

[0027] Two feature descriptors are encoded using convolutional layers, and feature information is extracted in different spatial dimensions using convolutional kernels of different sizes. and Two small convolutional kernels of different sizes form two different spatial attention mapping relationships. and This allows the spatial attention module to analyze the features of different regions; the corresponding expression is: (11) (12) in, and They respectively represent the use and Convolution is performed using convolution kernels. and These represent mean pooling and max pooling, respectively. Attention maps are concatenated using average channel pooling. and Spatial attention features are obtained through the Sigmoid activation function. The corresponding expression is: (13) Formula (11) is the derived mapping relationship of the spatial attention module.

[0028] Optical flow describes the motion patterns of image pixels over time. It models the changes in pixel intensity between adjacent frames to reveal the relative motion of objects or backgrounds, thus providing key information about dynamic scenes for the visual system. The attention mechanism in neural networks adjusts the weights at different locations or channels, making the network focus more on important features or regions, thereby improving the network's representational ability and performance. This mechanism is similar to human attention, enabling neural networks to process input data more effectively and improve the perception and utilization of key information. Adding attention modules at appropriate locations in the neural network model can enhance the network's feature extraction capabilities.

[0029] In one feasible embodiment, such as Figure 2 As shown, the present invention also provides a method for using a myopia prevention and coordinate correction device based on multimodal perception, comprising the following steps: S1: Device setup and initialization; After the user puts on the glasses, the system automatically calibrates the sensors and loads personal preference settings.

[0030] S2: Multimodal data acquisition and real-time analysis; The image module continuously captures facial data and analyzes attention metrics (such as pupil focus); the motion sensor monitors the sitting posture angle; and the light sensor collects ambient light data.

[0031] S3: Algorithm matching and state judgment; The algorithm engine integrates data from multiple sources to determine the user's current state (such as "fatigue", "poor posture", "insufficient lighting"); it dynamically adjusts thresholds based on historical data to achieve personalized monitoring.

[0032] S4: Intelligent Feedback and Intervention; If poor posture is detected and continues for more than a set time (e.g., 10 minutes), vibration is triggered and a voice prompt is given: "Please sit up straight." When eye fatigue is detected (e.g., blinking frequency is too low), the reminder frequency is automatically adjusted and a rest is suggested. When the ambient light is not up to standard, the app will notify parents or prompt users to improve the lighting.

[0033] S5: Data synchronization and parental involvement. All data is encrypted and synchronized to the parent's app, generating daily behavior reports including posture accuracy, attention span, and outdoor activity levels. Parents can set personalized rules and reward mechanisms. Through the above S1-S5 steps, this invention can achieve precise and personalized myopia prevention and posture correction, effectively improving the eye health of children and adolescents.

[0034] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limitations; therefore, other examples of exemplary embodiments may have different values.

[0035] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0036] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A myopia prevention and coordinate correction device based on multimodal perception, characterized in that, include: The multimodal perception unit includes an image recognition module, a motion sensor array, and an ambient light sensor. The image recognition module is used to capture the user's facial image in real time and uses a convolutional neural network to recognize and analyze the facial image to obtain the user's eye information. The motion sensor array is used to detect the user's head movement information and sitting posture information. The ambient light sensor is used to monitor the light intensity and color temperature information corresponding to the facial image in real time and evaluate the user's visual atmosphere based on the light intensity and color temperature information. An algorithm processing unit, connected to the multimodal perception unit, is used to run a lightweight deep learning model corresponding to the convolutional neural network and to perform fusion analysis on multi-source data to construct user behavior profile information. The multi-source data includes facial images, head movement information and sitting posture information, light intensity and color temperature information. The intelligent feedback unit includes a multi-mode reminder module and an atmosphere adaptation module. The multi-mode reminder module is used to receive the output of the algorithm processing unit and trigger interactive feedback information. The atmosphere adaptation module adjusts the user's visual environment according to the light intensity. The remote collaboration platform is used to receive and store the multi-source data, user behavior profile information and interactive feedback information, and generate corresponding personalized sitting posture rules.

2. The myopia prevention and coordinate correction device based on multimodal perception according to claim 1, characterized in that, The execution process of the image recognition module includes: Multiple facial images of the user are acquired, and the facial images are preprocessed to obtain a facial image. The preprocessing includes face alignment and size normalization. An optical flow estimation algorithm is used to analyze the pixel intensity of adjacent image frames of a facial image, and a two-dimensional displacement vector is used to represent dynamic changes to determine the motion vector of each pixel in each frame. The specific analysis process is as follows: In the preset expression sample frame at time t The pixel intensity at that location is After time The distance moved later becomes Its brightness value becomes Based on the principle of constant brightness between consecutive frames, the expression is: （1） Expanding and simplifying equation (1) using Taylor series, the corresponding optical flow equation is: （2） Remove general form Then substitute formula (1) into formula (2) and divide by . Formula (3) is obtained as follows: （3） in, The derivative is used to obtain the estimated vectors of horizontal and vertical optical flow velocity of micro-expressions by solving formula (3).

3. The myopia prevention and coordinate correction device based on multimodal perception according to claim 2, characterized in that, Also includes: Preset and The coordinates of the key points at the left and right inner corners of the eye are given. Using the midpoint of these two key points as the rotation center, the face alignment calculation process is as follows: （4） in, Horizontal angle of the face Rotated coordinates The facial region is accurately cropped from the original frame, and the facial region that best fits the face is obtained based on the main points around the multiple key points of the cropped area.

4. The myopia prevention and coordinate correction device based on multimodal perception according to claim 3, characterized in that, The facial image is analyzed using a convolutional neural network to obtain the user's eye information, including: The input to the pre-defined convolutional neural network is... Three-channel optical flow characteristic map The output will be labeled as positive, negative, and surprised. Feed in 24 sizes A regular convolution kernel with a size and stride of 2 is obtained. The shallowest feature F, whose size is ; Max pooling downsampling is performed on the shallowest feature F to reduce the feature dimensionality. The pooling downsampling operation is performed on the... Output of each pooling kernel for: （5） in, and These represent multiplicative factors and additive factors, respectively. This indicates a downsampling operation. Max pooling, on the other hand, uses a sliding window to calculate the maximum value in a specific region of the feature map; the corresponding expression is: （6） in, express medium size The area and The values ​​are 1, 2, and 3; The feature maps are reduced in dimensionality using global average pooling, and classified using a Softmax layer to map the information of each channel of the convolutional neural network.

5. The myopia prevention and coordinate correction device based on multimodal perception according to claim 1, characterized in that, Running the lightweight deep learning model corresponding to the convolutional neural network includes: A dual-attention module is used to enhance the network's ability to identify and analyze multi-source data, with an input size of... Feature mapping matrix Where C, H and W represent the number of channels, height and width of the feature mapping matrix, respectively, and the dual attention module includes the channel attention module CAM and the spatial attention module SAM; Output one-dimensional channel attention mapping matrix sequentially and two-dimensional attention mapping matrix Their sizes are respectively and The expression for the entire derivation process is as follows: （7） （8） in, express and as well as and Multiplying elements at the same position, For output features.

6. The myopia prevention and coordinate correction device based on multimodal perception according to claim 5, characterized in that, In the channel attention module, the spatial dimension needs to be reduced by 1 to generate a one-dimensional channel attention mapping matrix, including: For input features Simultaneously perform max pooling and average pooling operations to obtain different spatial feature descriptors. and ; The attention feature vectors of the two channels and The mixture is fused by addition. The sigmoid activation function is used to normalize the weights, and the attention feature matrix is ​​output. Broadcast it along the spatial dimensions to The specific calculation process is as follows: （9） in, This represents the normalized weights of the activation function. This indicates that a one-dimensional convolution kernel of length k is used for convolution, where the size of k is represented as: （10） Where C represents the number of channels in the feature map. It represents the odd number that is closest to it.

7. The myopia prevention and coordinate correction device based on multimodal perception according to claim 6, characterized in that, In the spatial attention module, the channel dimension needs to be reduced by 1 to generate a two-dimensional spatial attention mapping matrix, including: Input features in the spatial dimension Simultaneously perform max pooling and mean pooling operations to obtain different channel pooling feature descriptors. and ; Two feature descriptors are encoded using convolutional layers, and feature information is extracted in different spatial dimensions using convolutional kernels of different sizes. and Two small convolutional kernels of different sizes form two different spatial attention mapping relationships. and This allows the spatial attention module to analyze the features of different regions; the corresponding expression is: （11） （12） in, and They respectively represent the use and Convolution is performed using convolution kernels. and These represent mean pooling and max pooling, respectively. Attention maps are concatenated using average channel pooling. and Spatial attention features are obtained through the Sigmoid activation function. The corresponding expression is: （13） Formula (11) is the derived mapping relationship of the spatial attention module.

8. The myopia prevention and coordinate correction device based on multimodal perception according to claim 1, characterized in that, The motion sensor array includes an accelerometer and a gyroscope. The gyroscope and the accelerometer work together to detect the head tilt angle, head-down state, or head-tilt posture corresponding to the head motion information.

9. The myopia prevention and coordinate correction device based on multimodal perception according to claim 1, characterized in that, When the ambient light intensity is insufficient, the system can fine-tune the lens or remind the user to adjust their sitting position to optimize the visual environment.

Images