AI-based eye health detection system

Abstract

The invention relates to an eye health detection system based on AI, in particular to the field of eye health detection.Accurate screening is achieved through cross-modal cooperative processing, the system firstly eliminates audio and video signal time sequence deviation, and it is ensured that a voice instruction and facial expression are dynamically and accurately synchronized; then, the facial muscle motion features are fused to compensate noise interference, and the voice definition and the intention recognition accuracy are remarkably improved; the oral symptom description of the user is converted into a standardized index in combination with a medical knowledge base, and the confidence coefficient is dynamically corrected according to expression changes; and finally, whether re-questioning is needed or not is intelligently judged through a multi-mode contradiction detection mechanism, or a comprehensive eye health assessment report is generated, the whole set of scheme still keeps stable operation in a noisy family environment, the dependence of traditional equipment on a professional environment and operation skills is broken through, and non-professionals can also complete reliable eye health preliminary screening.

Description

Technical Field

[0001] This invention relates to the field of eye health detection, and more specifically, to an AI-based eye health detection system. Background Technology

[0002] Eye health screening plays a vital role in modern medical prevention systems. Currently, users generally rely on their home environment for self-testing, such as completing initial vision screening in the living room using smart devices. However, typical residential environments suffer from significant acoustic interference: television audio output is often in the 65-70 dB range, pet barking can reach up to 80 dB of sudden noise, and multi-person conversations create a wideband sound field superposition. These environmental factors, combined with user behavior characteristics (such as the volume of speech decreasing with distance and voice tremors in elderly users), result in the signal-to-noise ratio of the voice signal collected by the device microphone generally being below 10 dB. When the system guides users to perform interactive processes such as color vision testing through voice dialogue, the instruction distortion rate rises sharply, seriously affecting the accuracy of operations such as switching eye charts and recognizing astigmatism. At the same time, users' descriptions of visual symptoms are highly unstructured, such as describing "glare reaction" as "seeing rainbow halos around lights," or describing "distorted vision" as "straight lines becoming curved." Such colloquial expressions contain a large number of regional metaphors and subjective quantitative terms, and are often mixed with non-professional disease speculation information.

[0003] Traditional technical solutions face a dual technical bottleneck. In terms of noise suppression, existing speech recognition systems use recurrent neural network acoustic models for environmental noise reduction. However, the model training data is mostly concentrated in quiet laboratory environments (background noise <35dB), which is insufficient for generalizing to complex home sound fields. When the ambient noise is >60dB, the misrecognition rate of voice commands exceeds 30%, and it cannot adapt to sudden impulse noise. The acoustic front-end processing module does not effectively integrate multimodal auxiliary signals, such as the confused facial expressions and abnormally prolonged speech pauses when users perform color blindness tests, resulting in a lack of adaptability to dynamic environments. In terms of semantic parsing, general natural language processing tools rely on standard medical terminology databases, and the mapping accuracy for patients' commonly used descriptions such as "rainbow circles" and "mosquito shadows" is less than 45%. Although medical-specific models introduce disease association analysis, they violate the "non-diagnostic" principle of health screening systems and have a logical tendency to directly associate symptom descriptions with specific eye diseases. More importantly, existing technologies lack a quantitative distillation mechanism for unstructured expressions, and cannot extract quantifiable temporal frequency and spatial feature parameters from descriptions such as "worsening at night" and "more noticeable in the left eye," resulting in the loss of key health information. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing an AI-based eye health detection system that achieves accurate screening through cross-modal collaborative processing, thereby solving the problems mentioned in the background.

[0005] The technical solution of this invention to solve the above-mentioned technical problems is as follows: Specifically, it includes: a cross-modal spatiotemporal synchronization module, an expression-driven noise suppression module, a knowledge-enhanced semantic distillation module, and a multimodal confidence circuit breaker module; Cross-modal spatiotemporal synchronization module: When the timing deviation between the audio and video streams exceeds the threshold, the Hamming window function weighted signal of the audio frame and the facial ROI region features of the video frame are processed by dynamic sliding window integration to generate a timing-aligned synchronized audio stream and synchronized video stream. Expression-driven noise suppression module: Receives the synchronous audio stream and synchronous video stream, processes the speech domain features through a convolutional neural network, and performs weighted compensation by combining the facial action unit gradient features extracted by the graph neural network, and outputs a denoised speech signal, a user intent probability vector, and a frame-level expression feature change rate; Knowledge-enhanced semantic distillation module: After converting the denoised speech signal into text, it performs structured extraction through a bidirectional long short-term memory network and a conditional random field model combined with a medical symptom mapping dictionary. At the same time, it dynamically corrects the entity confidence based on the rate of change of facial expression features to generate standardized health indicator data. Multimodal confidence circuit breaker module: When the temporal derivative difference between the voice description feature vector and the facial expression physiological feature vector exceeds the circuit breaker threshold, the re-questioning protocol is activated; otherwise, the intent probability vector and standardized health indicator data are fused to generate an eye health assessment report.

[0006] In a preferred embodiment, the dynamic sliding window span generation process in the cross-modal spatiotemporal synchronization module is as follows: The system acquires the timestamp sequences of the original audio and video streams in real time, calculates the audio spectral entropy of adjacent timestamps and the rate of change of optical flow motion intensity of video frames. When the temporal deviation between the audio and video streams exceeds a threshold, an adaptive sliding window span parameter is generated by multiplying the ratio of the current audio spectral entropy value to the gradient norm of the video optical flow motion intensity by a window span scaling factor. The window span scaling factor is determined by the hardware type, with a value of 0.7 for mobile terminal devices and 0.8 for embedded devices. This sliding window span parameter is controlled within the range of 1 millisecond to 100 milliseconds, and frame offset caused by differences in device sampling rates is compensated by bilinear interpolation.

[0007] In a preferred embodiment, the joint processing of the Hamming window function weighted signal and facial ROI region features is as follows: An improved Hamming window function is used for windowing and framing of audio frames. The basic form of this window function is 0.54 minus 0.46 multiplied by 2πt divided by the cosine value of the sliding window span, and a nonlinear suppression gain coefficient controlled by the instantaneous spectral energy is superimposed. At the same time, an adaptive morphological RoI template is constructed for video frames. This template is generated by convolving the facial feature weight vector with the difference of Gaussian operator, and a sharpening coefficient is used to adjust the response intensity of facial key point features, thereby outputting the RoI heatmap layer. Perform the following operations during the spatiotemporal alignment phase: A1. Extract the real part spectral features of the short-time Fourier transform of the weighted audio frames, and the spatial gradient product features of the video RoI heatmap in the x and y directions. A2. Construct the above features into a two-dimensional feature matrix, and multiply it with the correlation coefficient matrix composed of audio-expression covariance factors to generate a cross-spectral density matrix, wherein the audio-expression covariance factors are dynamically fitted through statistical analysis of historical interaction data. A3. Calculate the optimal time correction by solving the minimum Frobenius norm residual between the matrix and the identity matrix under the time shift operator transformation; A4. Generate a spatiotemporal feature tensor by fusing and correcting audio and video data: Perform Kronecker product expansion operation on the spectral envelope features of the audio and the spatial gradient features of the video, and simultaneously superimpose the product of the time derivative of the cross spectral density matrix and the delay compensation matrix. A5. Decode and reconstruct the feature tensor into synchronized audio and video streams.

[0008] In a preferred embodiment, the operation of extracting facial motion unit gradient features using the joint graph neural network in the expression-driven noise suppression module specifically involves: A dynamic tracking model for key facial muscle groups was established, and calculations were performed separately for the eyebrow region, periorbital region, and lip region in the synchronized video stream. B1. Absolute value of the first partial derivative of the motion heatmap of the eyebrow region with respect to time; B2. The second-order mixed partial derivatives of the lip region heat map in the x and y directions in space, and the calculation results are converted into Frobenius norm form; The two types of features mentioned above are integrated into an initial gradient map through channel splicing, and then input into a pre-trained graph neural network topology for spatial aggregation. Meanwhile, in the audio branch, a five-layer convolutional kernel group is used to process Mel spectrum features, with convolutional kernel sizes of 3×3, 5×5, 3×3, 7×1 and 1×7, respectively. A 128-dimensional audio frequency domain feature tensor is generated through depthwise separable convolution and ReLU activation function. The final output is an eight-channel facial motion unit gradient tensor and an audio feature tensor.

[0009] In a preferred embodiment, the weighted compensation operation of the frequency domain features and the action unit features specifically involves: A dual-channel residual fusion architecture is constructed. The first channel applies an adaptive gating mechanism to the Mel spectrum features extracted by the convolutional neural network. The gating vector is generated by concatenating the feature vector after the gradient tensor of the action unit is transformed by the time-frequency domain projection and the audio spectrum features, and then by mapping through a 256-512 dimensional fully connected weight matrix and activating it with the Sigmoid function. The second channel multiplies the gate vector and the nonlinear hyperbolic tangent mapping result of the action unit gradient tensor after time-frequency domain projection transformation element by element, and then multiplies it by the weight value output by the dynamic decay function. The base of the decay function is 1.2 minus the sum of the absolute values ​​of the first norm of the action unit gradient tensor, which is 0.4 times. Frequency domain feature calibration is achieved by injecting the compensation amount output from the second channel into the residual connection point of the first channel. In the user intent probability vector generation stage, the temporal gradient of the compensated features is input into the long short-term memory network, and its output is multiplied by the exponential decay factor calculated based on the square of the second norm of the confused expression features to generate a six-dimensional intent probability distribution vector. At the same time, the temporal data stream of the change rate of the expression features is output synchronously, where the denominator of the decay factor contains the square value of the configurable inhibition coefficient. In the speech reconstruction stage, the denoised speech signal is reconstructed through the short-time Fourier inverse transform process: the phase angle information of the original audio features is preserved, and the compensated frequency domain features are multiplied element by element by the corresponding phase complex exponential function before performing the inverse spectral transform.

[0010] In a preferred embodiment, the cross-modal embedding operation of the denoised speech signal converted into text in the knowledge-enhanced semantic distillation module specifically includes: The system receives denoised speech signals and a medical symptom mapping dictionary as input. An end-to-end speech recognition model converts the denoised speech signals into word sequences, while simultaneously acquiring frame-level facial expression feature change rate data. A time alignment function maps the frame-level facial expression feature change rate into word-level facial expression feature vectors. A pre-trained language model in the medical field is used to process the word sequences to generate text embedding vectors. The text embedding vectors and word-level facial expression feature vectors are concatenated along the channel dimension to generate a joint embedding matrix that integrates text semantics and facial expression dynamic features. The time alignment operation uses a sliding window dynamic interpolation algorithm, calculating interpolation weights based on the offset between the speech recognition timestamp and the facial expression feature timestamp.

[0011] In a preferred embodiment, the specific operation of the structured extraction is as follows: A bidirectional long short-term memory network is constructed to process the fusion embedding vector sequence, generating forward and backward hidden state vectors respectively. The bidirectional hidden state vectors are concatenated and input into a fully connected layer to generate an emission score matrix. Entity transition constraints from a medical symptom mapping dictionary are introduced into the conditional random field decoding layer. The Viterbi algorithm is used to solve for the globally optimal entity label sequence, outputting a structured entity set containing symptom descriptions, entity types, and initial confidence levels.

[0012] In a preferred embodiment, the specific operation of dynamically correcting entity confidence based on the rate of change of facial expression features is as follows: Calculate the second norm of the acceleration of facial expression feature changes within the time period covered by each entity, and map it to an expression stability index through a trainable association matrix; compare the normalized difference between this index and the median stability index of the overall session, and convert the difference value into a confidence correction factor through an exponential decay function; multiply the initial confidence by the correction factor to generate a calibrated confidence, and filter out entities below the threshold. The specific operation of converting entity text into standard medical codes using a medical symptom mapping dictionary is as follows: apply the dictionary mapping function to each corrected entity to convert it into standard medical codes, and only retain entities with a confidence level greater than a set threshold. The specific operation of aggregating and generating standardized health indicator data streams by time window is as follows: Within a preset time window, entities of the type of symptoms are selected, each symptom entity is converted into a one-hot vector in the symptom coding space according to its standard medical code, and multiplied by the calibrated confidence of the entity. All weighted one-hot vectors within the window are summed to generate the standardized health indicator vector for that time window. Finally, the standardized health indicator vector sequence arranged in chronological order is output as the standardized health indicator data stream.

[0013] In a preferred embodiment, the operation of detecting the temporal derivative difference between the speech description feature vector and the facial expression physiological feature vector in the multimodal confidence circuit breaker module specifically involves: Real-time acquisition of user intent probability vectors, standardized health indicator data streams, and temporal derivatives of facial physiological features as input data; First, semantic feature extraction is performed: the temporal derivatives of the three types of symptom subvectors (refractive, color vision, and visual field) in the standardized health indicator data stream are weighted and summed using the symptom-intent association matrix, and then activated by a normalized exponential function to generate semantic dynamic feature vectors. Next, perform the contradiction measurement operation: calculate the logarithmic ratio of the user intent probability vector to the semantic dynamic feature vector, superimpose a preset zero-prevention constant, and apply predefined masking rules to filter non-contradiction sensitive dimensions; input the filtered vector into the weighted singular value norm processor, and output the non-negative contradiction index value.

[0014] In a preferred embodiment, the operation of activating the re-questioning protocol and generating the eye health assessment report specifically includes: Set a dynamic circuit breaker threshold parameter, which is generated by a linear combination of the base threshold and the absolute value of the second time derivative of the facial physiological feature vector; When the contradiction index exceeds the dynamic circuit breaker threshold and the confusion dimension component of the user intent probability vector is greater than 0.5, the reinforcement learning decision protocol is triggered: based on the current user intent state, contradiction index and facial expression physiological characteristics, a re-questioning strategy is selected from the action space and sent to the user. Otherwise, the standardized health indicator data stream is subjected to time dimension maximum pooling processing, which is then concatenated with the user intent probability vector to form a joint feature. After linear transformation by the weight matrix, the final eye health assessment report is generated.

[0015] The beneficial effects of this invention are as follows: Accurate screening is achieved through cross-modal collaborative processing. The system first eliminates timing deviations in audio and video signals, ensuring precise synchronization between voice commands and facial expressions. Then, it integrates facial muscle movement features to compensate for noise interference, significantly improving voice clarity and intent recognition accuracy. Next, it combines a medical knowledge base to transform users' colloquial symptom descriptions into standardized indicators and dynamically adjusts confidence levels based on facial expression changes. Finally, a multimodal contradiction detection mechanism intelligently determines whether to ask questions again or generates a comprehensive eye health assessment report. The entire solution maintains stable operation even in noisy home environments, breaking through the dependence of traditional equipment on professional environments and operating skills, enabling non-professionals to complete reliable initial eye health screenings. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a block diagram of the system structure of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0019] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0020] This embodiment provides, for example Figure 1-2 The illustrated AI-based eye health monitoring system specifically includes: Cross-modal spatiotemporal synchronization module, facial expression-driven noise suppression module, knowledge-enhanced semantic distillation module, and multimodal belief circuit breaker module; Cross-modal spatiotemporal synchronization module: When the timing deviation between the audio and video streams exceeds the threshold, the Hamming window function weighted signal of the audio frame and the facial ROI region features of the video frame are processed by dynamic sliding window integration to generate a time-aligned synchronized audio stream and synchronized video stream. This module achieves sub-millisecond precise alignment through dynamic spectral analysis and joint integration of the feature domain, eliminating the modal fragmentation problem caused by microphone start-up delay and camera exposure jitter. The expression-driven noise suppression module receives synchronized audio and video streams, processes speech domain features through a convolutional neural network, and performs weighted compensation using facial action unit gradient features extracted by a graph neural network. It outputs a denoised speech signal, a user intent probability vector, and frame-level expression feature change rates. After the cross-modal spatiotemporal synchronization module outputs aligned synchronized audio and video streams, traditional solutions typically process the dual-modal signals independently, which can easily lead to a disconnect between noise suppression and intent recognition. Therefore, this module introduces facial action unit gradient features as the neural modulation signal for audio frequency domain compensation and achieves joint optimization of biological behavioral features and physical acoustic features through a dual-channel residual network, overcoming the 15dB signal-to-noise ratio bottleneck of medical interaction systems in noisy environments. Knowledge-enhanced semantic distillation module: After converting the denoised speech signal into text, it performs structured extraction by combining a bidirectional long short-term memory network with a conditional random field model and a medical symptom mapping dictionary. At the same time, it dynamically corrects the entity confidence based on the rate of change of facial expression features to generate standardized health indicator data. Multimodal confidence circuit breaker module: When the temporal derivative difference between the voice description feature vector and the facial expression physiological feature vector exceeds the circuit breaker threshold, the re-questioning protocol is activated; otherwise, the intent probability vector and standardized health indicator data are fused to generate an eye health assessment report.

[0021] In this embodiment, it is specifically necessary to explain the cross-modal spatiotemporal synchronization module. The dynamic sliding window span generation process in the dynamic sliding window integration processing is as follows: Real-time acquisition of the timestamp sequence of the raw audio stream and Timestamp sequence of the original video stream Calculate the audio spectral entropy and the rate of change of optical flow motion intensity of video frames for adjacent timestamps; when the timing deviation between the audio and video streams exceeds a threshold, the timing deviation threshold is calculated using the following formula: ; in, This represents the hardware latency baseline coefficient (mobile phone = 1.8, embedded device = 3.2), used to characterize the device's data acquisition latency characteristics and correct for time deviations in different hardware. The audio sampling rate (the number of audio samples collected per second, such as 44.1kHz) is used to determine the temporal resolution of the audio signal. Indicates the video frame rate (the number of video frames captured per second, such as 30fps), used to determine the temporal resolution of the video stream. This represents the timing deviation threshold, a critical value used to determine whether audio and video streams need alignment. It is generated by multiplying the ratio of the current audio spectral entropy value to the gradient norm of the video optical flow motion intensity by a window scaling factor. Its expression is: ; in, This represents the adaptive sliding window span, used to dynamically adjust the analysis window width and optimize alignment accuracy. Its value range is: when ,otherwise (Minimum 100ms or 1 frame duration) This represents the window scaling factor (mobile = 0.7, embedded = 0.8), used to adjust the sensitivity of the sliding window to acoustic features. The audio spectral entropy is used to quantify the complexity of a speech signal (a higher value indicates greater environmental noise complexity). It is calculated by performing an FFT on the audio frame and then calculating the Shannon entropy. , Represents discrete frequency points (single frequency components obtained after decomposition by Fast Fourier Transform (FFT)). Represents frequency probability distribution ( The normalized power spectrum energy percentage at a given frequency, with a value range of [0,1], is calculated as follows: , Represents the power spectral density of a signal (at frequency). (Power of the signal at the location) This represents the total power of the entire spectrum, i.e., the total energy of the signal. It represents the norm of optical flow intensity gradient between video frames, used to measure the magnitude of motion changes between adjacent video frames (the larger the value, the more drastic the user's movement). It represents the gradient of optical flow changes (displacement vector field) between adjacent video frames. This represents the measured timing deviation, i.e., the actual detected difference in audio and video timestamps. It is calculated as follows: (Difference between audio and video frame timestamps); where the window span scaling factor is determined by the hardware type, with a value of 0.7 for mobile terminal devices and 0.8 for embedded devices; this sliding window span parameter is controlled in the range of 1 millisecond to 100 milliseconds, and frame offset caused by differences in device sampling rate is compensated by bilinear interpolation; The joint processing of the Hamming window function weighted signal and facial ROI region features is as follows: The audio frames are windowed and framed using an improved Hamming window function. The calculation method is as follows: ; This represents a weighted audio frame (the output result after windowing the original audio signal), t represents the time-domain function of the signal as a function of time, and the subscript win indicates the "windowing" operation. This represents the raw audio frame (the unprocessed audio signal input to the system). This represents the improved Hamming window function, whose basic form is 0.54 minus 0.46 multiplied by 2πt divided by the cosine of the sliding window span, and superimposed with a nonlinear suppression gain coefficient controlled by the instantaneous spectral energy. Its expression is: ; Where t represents the audio frame time variable, and its value ranges from [0, ..., t]. ], This represents the span of the dynamic sliding window generated in step 1, with a value range of [1, 100] ms. This represents the instantaneous spectral energy of the audio signal, used to reflect the intensity of the audio signal at the current moment. This represents the nonlinear control factor (set to 0.3), used to adjust the intensity of the nonlinear influence of instantaneous energy on the window function (the larger the value, the steeper the suppression). This represents the gain suppression factor (set to 1.2), used to control the gain attenuation of non-stationary noise (>1 indicates suppression). The function of this formula is: through instantaneous energy... Dynamically suppress sudden noise and improve speech signal clarity; simultaneously, construct an adaptive morphological RoI template for video frames. This template is generated by convolving the facial feature weight vector with the difference of Gaussian operator, and is supplemented by a sharpening coefficient to adjust the response intensity of facial key point features. Its expression is: ; Where (x,y) represents the image space coordinates, used to define the spatial location of the facial region. This represents the facial feature weight vector (assigning weights to different facial regions, such as...). , , , , The range of values ​​is , Represents the Gaussian filter kernel (standard deviation) ), smooth facial features, The larger the value, the higher the ambiguity; its range is... , This represents the template sharpening factor (set to 1.6), used to control the degree of sharpening in morphological operations (the higher the value, the sharper the edges). This represents a morphological dilation operation used to enhance the connectivity of salient regions. This represents a multi-scale morphological gradient operator (where k is the structuring element size) used to extract gradient features of facial ROIs. This represents the difference Gaussian operator, used to enhance edge features (by subtracting two Gaussian kernels of different scales). This represents the raw video stream data, which is then used to output the RoI heatmap. ; Perform the following operations during the spatiotemporal alignment phase: A1. Extract the real part spectral features of the short-time Fourier transform of the weighted audio frame. (The weighted audio signal after noise reduction), and the video RoI heatmap layer. (A 128×128 matrix, representing the response intensity of key facial regions) spatial gradient product features in the x and y directions; A2. Construct the above features into a two-dimensional feature matrix, and multiply it by the correlation coefficient matrix composed of audio-expression covariance factors to generate a cross-spectral density matrix, the expression of which is: ; in, This represents the weighted audio frame signal (audio time-domain data processed by the improved Hamming window). This represents a facial RoI heatmap, characterizing the dynamic features of key facial regions. This represents the short-time Fourier transform, used to convert audio signals into frequency domain complex signals. This represents the extraction of the real part of a complex signal, used to extract physically interpretable frequency domain features. represents the gradient operator in the x / y direction, used to extract facial expression motion vectors; 'o' represents the Hadamard product (element-wise multiplication), used to fuse bidirectional motion features. The audio-expression covariance factor is used to quantify the correlation strength between speech and facial expressions. Its value ranges from [−0.8, 0.8]. The audio-expression covariance factor is dynamically fitted through statistical analysis of historical interaction data. A3. The optimal time correction is calculated by solving for the minimum Frobenius norm residual between the matrix and the identity matrix under the time-shift operator transformation. The calculation method is as follows: ; in, This represents the time shift operator matrix, used to simulate time offset. The linear transformation of , where I represents the identity matrix (the ideally aligned reference target). The Frobenius norm is used to measure the degree of difference between matrices. This represents the time correction amount (output multimodal alignment offset value), and its value ranges from [−20, 20] ms; A4. Generating a spatiotemporal feature tensor by fusing and correcting the audio and video data: The spectral envelope features of the audio and the spatial gradient features of the video are expanded using a Kronecker product, and then the product of the time derivative of the cross-spectral density matrix and the delay compensation matrix is ​​superimposed. The expression for the spatiotemporal feature tensor is: ; z represents the higher-order feature tensor (dimension: 8192×8192) after multimodal fusion, used to reconstruct the synchronized data. This represents the spectral energy distribution characteristics extracted from the corrected audio using the short-time Fourier transform (dimension: 64×1). This represents the product of the partial derivatives of the facial RoI heatmap in the x and y directions (dimension: 128×128), characterizing changes in facial micro-expressions. Representing the tensor product expansion operator: Integrate audio and video features in a higher dimension (physical meaning: establish a fully connected relationship between cross-modal features). This represents the rate of change matrix of cross-modal covariance within adjacent time windows (physical meaning: capturing the dynamic correlation between acoustic features and facial movements). This represents the standard matrix multiplication operator. Represents a diagonal matrix diagonal elements Hardware channel latency compensation coefficient (e.g., on mobile devices) Embedded devices ); A5. Decode and reconstruct the feature tensor into a synchronized audio stream. With video stream .

[0022] In this embodiment, the expression-driven noise suppression module, specifically the operation of extracting gradient features of facial action units using a graph neural network, is as follows: A dynamic tracking model for key facial muscle groups was established, and calculations were performed separately for the eyebrow region, periorbital region, and lip region in the synchronized video stream. B1. Absolute value of the first partial derivative of the motion heatmap of the eyebrow region with respect to time; B2. The second-order mixed partial derivatives of the lip region heat map in the x and y directions in space, and the calculation results are converted into Frobenius norm form; The two types of features mentioned above are integrated into an initial gradient map through channel splicing, and then input into a pre-trained graph neural network topology for spatial aggregation. Meanwhile, in the audio branch, a five-layer convolutional kernel group is used to process Mel-frequency spectral features, with kernel sizes of 3×3, 5×5, 3×3, 7×1, and 1×7. A 128-dimensional audio frequency domain feature tensor is generated through depthwise separable convolution and the ReLU activation function, and its expression is as follows: ; This represents the parameters of the convolution kernel group. * indicates depthwise separable convolution. By combining grouped convolution with pointwise convolution, the computational load is reduced (saving 80% of computational resources compared to standard convolution). This represents the logarithmic Mel spectrum, and its functions are: 1. To extract the Mel spectrum by windowing and framing audio; 2. To take the logarithm of the energy. , Represents the linear rectification activation function. Represents the audio frequency domain feature tensor (outputs a 128-dimensional feature vector). The final outputs are an eight-channel facial action unit gradient tensor (representing muscle movement dynamics) and an audio feature tensor (representing speech spectral characteristics). The expression for the eight-channel action unit gradient tensor is as follows: ; in, This represents a heat map of eyebrow muscle movement (motor energy distribution in the brow arch region (frontalis / corrugator supercilii) (intensity value range [0,255])). This represents the absolute value of the partial derivative in the time direction. This represents a heat map of motion in the lip region (distribution of energy in the orbicularis oris muscle). The spatial mixed partial derivative operator is used to detect coordinated muscle movements (such as the horizontal and vertical linkage of lips during speech). The Frobenius norm is used to convert a mixed partial derivative matrix into scalar features. This indicates a channel splicing operation used to integrate the temporal derivative of the eyebrow and the spatial derivative of the lip (outputting a four-channel feature map). The graph neural network spatial aggregation (nodes are 26 key points on the face, edges are muscle movement correlations, and the final output is an eight-channel facial motion unit gradient tensor). The gradient tensor of the action unit (the eight-channel muscle dynamics feature specifically includes the following eight types of muscles: frontalis contraction, decrement of interbrevis strength, orbicularis oculi closure, nasal ala elevation, levator labii superioris contraction, zygomaticus major activity, orbicularis oris opening and closing, and platysma depression) represents the action unit gradient tensor (the eight-channel muscle dynamics feature specifically includes the following eight types of muscles: frontalis contraction, decrement of interbrevis strength, orbicularis oculi closure, nasal ala elevation, levator labii superioris contraction, zygomaticus major activity, orbicularis oris opening and closing, and platysma depression). The weighted compensation operation for frequency domain features and action unit features is as follows: A dual-channel residual fusion architecture is constructed. The first channel applies an adaptive gating mechanism to the Mel spectrum features extracted by the convolutional neural network. This gating vector is generated by concatenating the feature vector of the action unit gradient tensor after time-frequency domain projection transformation with the audio spectrum features, and then generating it by mapping through a 256x512 dimensional fully connected weight matrix and activating it with the Sigmoid function. The expression for the gating vector is: ; in, This represents the Sigmoid activation function. This represents a 256×512 dimensional weight matrix. Represents the audio frequency domain feature tensor (dimension: 128). This represents the gradient tensor of an eight-channel facial motion unit. This indicates a feature concatenation operation. Represents the gate vector (dimension: 256); The second channel performs element-wise multiplication of the gate vector and the nonlinear hyperbolic tangent mapping result of the action unit gradient tensor after time-frequency domain projection transformation, and then multiplies it by the weight value output by the dynamic decay function. The decay function has a base of 1.2 and subtracts the sum of the absolute values ​​of the first norm of the action unit gradient tensor by 0.4. The expression for the compensation output by the second channel is as follows: ; in, This represents the time-frequency domain projection transformation (mapping 128×128 video features to a 128-Vimel band). This represents element-wise multiplication. Indicates the dynamic decay factor. , express Norm (the sum of the absolute values ​​of the features of each channel). Indicates the amount of biological behavioral compensation (dimension: 128); The compensation amount from the second channel is injected into the residual connection point of the first channel to achieve frequency domain feature calibration. The expression for the compensated feature tensor is: ; in, This represents the frequency domain characteristics after compensation (dimension: 128). In the user intent probability vector generation stage, the temporal gradient of the compensated features is input into the Long Short-Term Memory network, and its output is multiplied by an exponential decay factor calculated based on the squared second-order norm of the confused expression features. The intent probability vector formula is: ; in, This represents the frequency domain characteristic matrix after compensation. This represents the partial derivative operator in the time direction. The complex features of a confused expression Represents the gradient tensor of the action unit, and confuse represents the combined features of the frontalis muscle and orbicularis oculi muscle. express Norm square (sum of squares of vector elements). The coefficient representing the expression of confusion is set to 0.7, and exp(·) represents the natural logarithm function. Representing a six-dimensional intent probability vector, generating a six-dimensional intent probability distribution vector, and simultaneously outputting a temporal data stream of facial expression feature change rates. The calculation method is as follows: ,in, This represents a 256×512 dimensional weight matrix. Represents the audio frequency domain feature tensor (dimension: 128). This represents the function that decays the amplitude of muscle movement. The denominator of the attenuation factor includes a configurable squared value of the suppression coefficient; In the speech reconstruction stage, the denoised speech signal is reconstructed through a short-time Fourier inverse transform process: the phase angle information of the original audio features is preserved, and the compensated frequency domain features are multiplied element-wise by the corresponding phase complex exponential function before performing an inverse spectral transform. The spectral reconstruction formula is as follows: ; in, This represents the Mel-spectral characteristics of the original audio. This indicates the extraction of complex phase angles. Construction of complex exponential functions: o represents the Hadamard product (element-wise matrix multiplication), and ISTFT represents the inverse short-time Fourier transform. Inverse Fourier transform framework This represents the reconstructed, denoised speech signal.

[0023] In this embodiment, the knowledge-enhanced semantic distillation module specifically describes the cross-modal embedding operation after the denoised speech signal is converted into text: The system receives a denoised speech signal and a medical symptom mapping dictionary as input, and then processes the denoised speech signal through an end-to-end speech recognition model. (T represents the number of sampling points) Convert to a word sequence ( Let N represent the i-th word after text segmentation, and N represent the total number of words. The expression for speech transcription is: ; Here, STT(·) represents an end-to-end speech recognition model (such as Conformer-Transducer) used to achieve end-to-end audio-to-text conversion, while acquiring frame-level facial expression feature change rate data. A time alignment function is used to map the frame-level facial expression feature change rate to a word-level facial expression feature vector. A pre-trained language model from the medical field is used. The word sequence is processed to generate a text embedding vector. The text embedding vector is then concatenated with the word-level expression feature vector along the channel dimension. The expression for cross-modal embedding is: ; in, This represents a BERT model fine-tuned for the medical field, used to understand the contextual semantics of specialized medical terminology. This represents the initial embedding of the text (the vectorized representation of lexical units in the medical semantic space). This represents the time alignment function, used to adjust the rate of change of frame-level facial features. Mapping to the lexical level , This represents lexical-level facial expression features (facial dynamics matched with lexical time windows), where d represents the lexical embedding dimension. This represents the frame-level rate of change of facial expression features (an indicator of instantaneous changes in facial muscle movement), and its value range is... (k represents the number of facial expression features) This represents a vector concatenation operation that generates a joint embedding matrix that integrates textual semantics and facial expression dynamic features. The time alignment operation employs a sliding window dynamic interpolation algorithm, which calculates interpolation weights based on the offset between the speech recognition timestamp and the facial expression feature timestamp, ensuring that each word embedding vector fuses the corresponding time period's dynamic facial expression features. The specific operations of structured extraction are as follows: A bidirectional long short-term memory network is constructed to process the fused embedding vector sequence, generating forward hidden state vectors and backward hidden state vectors respectively. Their expressions are as follows: ; ; in, This represents the fused embedding vector of the nth word (a joint feature representation of text and emoji). This represents the forward hidden state of the nth lexical unit, i.e., the forward context dependency of the captured lexical unit. This represents the inverse hidden state of the nth lexical, i.e., the inverse context dependency of the captured lexical; the bidirectional hidden state vectors are concatenated and input into the fully connected layer to generate the emission score matrix, the expression of which is: ; in, This represents the bidirectional state concatenation (complete context feature representation) of the nth word. The hidden layer dimension of the LSTM (set to 256) is used to control the information capacity of the state vector; entity transition constraints based on a medical symptom mapping dictionary are introduced into the conditional random field decoding layer to prohibit illegal state transitions between symptom types and body parts; the globally optimal entity label sequence is solved using the Viterbi algorithm, with the following formula: ; in, This represents the label probability (predicted entity type) of the nth word, and its value ranges from [0,1]. Indicates by Emission score, used to evaluate the matching degree between lexical features and labels. This refers to a multilayer perceptron (a computational unit that generates emission scores). This represents the transition score, used to evaluate the legality of the label sequence (e.g., prohibiting illegal transitions from "degree to location"). This represents the predicted label (entity type label (symptom / location / degree)) for the nth term, where N represents the total number of terms (sentence length parameter). The output is a structured set of entities containing symptom descriptions, entity types, and initial confidence scores. , This represents the initial set of symptom entities (containing all identified symptom entities). This represents the text of the m-th entity (the original text describing the patient's symptoms). This represents the type of the m-th entity (the type classification of symptoms). Let represent the initial confidence level (entity identification reliability) of m entities, with a value range of [0,1], and calculated as follows: , It represents the two-way state of the entity's central word (representing the entity's core features). This represents the confidence weight vector (parameters used to map features to confidence levels). This represents the Sigmoid activation function, used to compress scores into probability values; The specific operation of dynamically adjusting entity confidence based on the rate of change of facial expression features is as follows: The second norm of the acceleration of facial feature changes within the time period covered by each entity is calculated using the following formula: ; in, This represents the acceleration of facial expression changes, used to capture abrupt changes in facial muscle movement. The calculation method is as follows: , This indicates that the entity covers the lexical index set (entity) The location range in the text is used to define the time period corresponding to the facial expression features. This represents the number of entity persistence lexical units, used to calculate the window average stability. This represents a local stability index used to quantify the consistency of facial expressions across different time periods. It is mapped to an expression stability index via a trainable association matrix, and its expression is: ; Where M represents the trainable correlation matrix ( A parameter matrix (where k represents the number of facial expression features) is used to learn the mapping relationship between muscle movement and semantic reliability. This index is then compared with the median stability index of the overall conversation using a normalized difference analysis. The difference value is converted into a confidence correction factor using an exponential decay function. The confidence recalibration formula is as follows: ; in, Represents the median of global stability (i.e. ), used to provide a session-level reference baseline, Denotes the Frobenius norm. Represents a minimum constant (whose value is set to 0). ), used to prevent division by zero errors, This represents the correction intensity factor (set to 0.3), used to control the correction magnitude. This represents the corrected confidence level, with a value range of [0,1]. The initial confidence level is multiplied by the correction factor to generate the calibrated confidence level. And filter out entities below the threshold; The specific steps for converting entity text into standard medical codes using a medical symptom mapping dictionary are as follows: Apply the dictionary mapping function to each corrected entity to convert it into standard medical codes, and only retain entities with a confidence level greater than a set threshold. The semantic distillation formula is: ; in, It represents a standardized set of entities (including encoding, type, and confidence level). Map represents the operation of converting terms to standard encodings (e.g., "headache" → "MA82.0"). This represents a medical symptom mapping dictionary (a predefined medical terminology mapping library (key: spoken description, value: standard encoding)). This represents a dictionary mapping function used to convert spoken descriptions into standardized medical codes that can be processed by machines. For example, if the input is "seeing shadows", the output is "H53.4" (ICD-10 encoding). This represents the symptom entity text, i.e. the content of the symptom described by the user (such as "seeing double"). Indicates the entity type, i.e., the entity classification label (limited values: symptoms / location / severity). This represents the confidence score after correction based on facial expression features (combining the user's facial micro-expression dynamic calibration to describe confidence), and its value range is [0,1] (e.g., 0.83). This represents the confidence filtering threshold (set to 0.6), used to prevent noisy data with low confidence from entering the aggregation process; The specific operation for aggregating and generating standardized health indicator data streams by time window is as follows: Within a preset time window, entities of type symptom are selected. Each symptom entity is converted into a one-hot vector in the symptom coding space according to its standard medical code, and multiplied by the calibrated confidence of the entity. All weighted one-hot vectors within the window are summed to generate the standardized health indicator vector for that time window. The aggregation formula is as follows: ; ; in, This represents the symptom quantification results for the j-th time window. This represents a weighted summation operation on symptom entities, quantifying the severity of symptoms within a specific time period. Indicates the aggregation period (e.g., 30 seconds). This indicates the filtering criteria, which only process symptom-related entities and exclude descriptions of location / severity. Indicates the time of occurrence of the symptom description, and S represents the satisfaction of the condition. and A subset of conditional symptom entities This indicates that the current symptom location is marked in the complete set of standard encodings v; the final output is a sequence of standardized health indicator vectors arranged in chronological order as a standardized health indicator data stream. Where O represents the standardized health indicator data stream, This represents the quantification results of symptoms within a single time window.

[0024] In this embodiment, it is specifically necessary to explain the multimodal confidence circuit breaker module. The operation of detecting the temporal derivative difference between the speech description feature vector and the facial expression physiological feature vector is as follows: Real-time acquisition of user intent probability vectors, standardized health indicator data streams, and temporal derivatives of facial physiological features as input data; First, semantic feature extraction is performed: the temporal derivatives of the three symptom subvectors (refractive, color vision, and visual field) in the standardized health indicator data stream are weighted and summed using the symptom-intent association matrix, and then activated by a normalized exponential function to generate semantic dynamic feature vectors. The semantic feature extraction formula is: ; in, Representing the symptom-intent association matrix, This represents the subvector of the k-th type of symptom (refractive / color vision / visual field). Represents a semantic dynamic feature vector; Next, the contradiction measurement operation is performed: the logarithmic ratio of the user intent probability vector to the semantic dynamic feature vector is calculated, and after superimposing a preset zero-prevention constant, predefined masking rules are applied to filter non-contradiction-sensitive dimensions; the filtered vector is then input into the weighted singular value norm processor. The contradiction measurement calculation formula is as follows: ; Where k represents the contradiction index. Represents logarithmic ratio operations. This indicates that non-contradiction-sensitive dimensions (such as "system error") are masked, and , Denotes the weighted singular value norm ( ), This indicates the zero constant (whose value is set to zero). Output the non-negative contradiction index value k; The specific steps for activating the re-questioning protocol and generating an eye health assessment report are as follows: A dynamic circuit breaker threshold parameter is set, which is generated by a linear combination of the base threshold and the absolute value of the second time derivative of the facial physiological feature vector. The formula for calculating the dynamic circuit breaker threshold is as follows: ; in, Indicates the dynamic circuit breaker threshold. This represents the base threshold (set to 0.7). This represents the sensitivity coefficient to facial expression fluctuations (its value is set to 0.2). Acceleration operators representing physiological characteristics; When the contradiction index exceeds the dynamic circuit breaker threshold and the confusion dimension component of the user intent probability vector is greater than 0.5, its expression is: ; in, This indicates the probability of user confusion (>0.5 triggers the protection mechanism). This represents the circuit breaker flag, with a binary output: 0 = no circuit breaker, 1 = circuit breaker triggered. The reinforcement learning decision protocol is triggered based on the current user intent state, contradiction index, and facial physiological characteristics. A re-questioning strategy is selected from the action space and sent to the user. Its expression is: ; in, Represents the state vector. A represents the action space {simplifying the problem, switching the question type, and adding visual cues}, and Q(·) represents the DQN network parameters. Dynamic updates This indicates time-dimension pooling. This represents the optimal action (reinforcement learning decision output). Otherwise, perform time-dimension max pooling on the standardized health indicator data stream, concatenate it with the user intent probability vector to form a joint feature, and generate the final eye health assessment report after linear transformation of the weight matrix. The expression for this report is: ; in, This indicates that the symptom indicators are time-pooled, and , Indicates the report generation weights. This indicates the output report (eye health assessment report or re-question instruction).

[0025] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0026] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0027] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0028] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0029] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0030] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0031] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An AI-based eye health detection system, characterized in that, Specifically, it includes: Cross-modal spatiotemporal synchronization module, facial expression-driven noise suppression module, knowledge-enhanced semantic distillation module, and multimodal belief circuit breaker module; Cross-modal spatiotemporal synchronization module: When the timing deviation between the audio and video streams exceeds the threshold, the Hamming window function weighted signal of the audio frame and the facial ROI region features of the video frame are processed by dynamic sliding window integration to generate a timing-aligned synchronized audio stream and synchronized video stream. Expression-driven noise suppression module: Receives the synchronous audio stream and synchronous video stream, processes the speech domain features through a convolutional neural network, and performs weighted compensation by combining the facial action unit gradient features extracted by the graph neural network, and outputs a denoised speech signal, a user intent probability vector, and a frame-level expression feature change rate; Knowledge-enhanced semantic distillation module: After converting the denoised speech signal into text, it performs structured extraction through a bidirectional long short-term memory network and a conditional random field model combined with a medical symptom mapping dictionary. At the same time, it dynamically corrects the entity confidence based on the rate of change of facial expression features to generate standardized health indicator data. Multimodal confidence circuit breaker module: When the temporal derivative difference between the voice description feature vector and the facial expression physiological feature vector exceeds the circuit breaker threshold, the re-questioning protocol is activated; otherwise, the intent probability vector and standardized health indicator data are fused to generate an eye health assessment report.

2. The AI-based eye health detection system according to claim 1, characterized in that: In the cross-modal spatiotemporal synchronization module, the dynamic sliding window span generation process in the dynamic sliding window integration processing is as follows: The timestamp sequences of the original audio stream and the original video stream are acquired in real time, and the audio spectral entropy and the rate of change of optical flow motion intensity of video frames at adjacent timestamps are calculated. When the temporal deviation between the audio and video streams exceeds the threshold, an adaptive sliding window span parameter is generated by multiplying the ratio of the current audio spectral entropy value to the gradient norm of the video optical flow motion intensity by the window span scaling factor. The window span scaling factor is determined by the hardware type, with a value of 0.7 for mobile terminal devices and 0.8 for embedded devices. This sliding window span parameter is controlled within the range of 1 millisecond to 100 milliseconds, and frame offset caused by differences in device sampling rates is compensated by bilinear interpolation.

3. The AI-based eye health detection system according to claim 2, characterized in that: The joint processing of the Hamming window function weighted signal and facial ROI region features is as follows: An improved Hamming window function is used for windowing and framing of audio frames. The basic form of this window function is 0.54 minus 0.46 multiplied by 2πt divided by the cosine value of the sliding window span, and a nonlinear suppression gain coefficient controlled by the instantaneous spectral energy is superimposed. At the same time, an adaptive morphological RoI template is constructed for video frames. This template is generated by convolving the facial feature weight vector with the difference of Gaussian operator, and a sharpening coefficient is used to adjust the response intensity of facial key point features, thereby outputting the RoI heatmap layer. Perform the following operations during the spatiotemporal alignment phase: A1. Extract the real part spectral features of the short-time Fourier transform of the weighted audio frames, and the spatial gradient product features of the video RoI heatmap in the x and y directions. A2. Construct the above features into a two-dimensional feature matrix, and multiply it with the correlation coefficient matrix composed of audio-expression covariance factors to generate a cross-spectral density matrix, wherein the audio-expression covariance factors are dynamically fitted through statistical analysis of historical interaction data. A3. Calculate the optimal time correction by solving the minimum Frobenius norm residual between the matrix and the identity matrix under the time shift operator transformation; A4. Generate a spatiotemporal feature tensor by fusing and correcting audio and video data: Perform Kronecker product expansion operation on the spectral envelope features of the audio and the spatial gradient features of the video, and simultaneously superimpose the product of the time derivative of the cross spectral density matrix and the delay compensation matrix. A5. Decode and reconstruct the feature tensor into synchronized audio and video streams.

4. The AI-based eye health detection system according to claim 3, characterized in that: In the expression-driven noise suppression module, the operation of extracting gradient features of facial action units using the joint graph neural network is specifically as follows: A dynamic tracking model for key facial muscle groups was established, and calculations were performed separately for the eyebrow region, periorbital region, and lip region in the synchronized video stream. B1. Absolute value of the first partial derivative of the motion heatmap of the eyebrow region with respect to time; B2. The second-order mixed partial derivatives of the lip region heat map in the x and y directions in space, and the calculation results are converted into Frobenius norm form; The two types of features mentioned above are integrated into an initial gradient map through channel splicing, and then input into a pre-trained graph neural network topology for spatial aggregation. Meanwhile, in the audio branch, a five-layer convolutional kernel group is used to process Mel spectrum features, with convolutional kernel sizes of 3×3, 5×5, 3×3, 7×1 and 1×7, respectively. A 128-dimensional audio frequency domain feature tensor is generated through depthwise separable convolution and ReLU activation function. The final output is an eight-channel facial motion unit gradient tensor and an audio feature tensor.

5. The AI-based eye health detection system according to claim 4, characterized in that: The weighted compensation operation of the frequency domain features and action unit features is specifically as follows: A dual-channel residual fusion architecture is constructed. The first channel applies an adaptive gating mechanism to the Mel spectrum features extracted by the convolutional neural network. The gating vector is generated by concatenating the feature vector after the gradient tensor of the action unit is transformed by the time-frequency domain projection and the audio spectrum features, and then by mapping through a 256-512 dimensional fully connected weight matrix and activating it with the Sigmoid function. The second channel multiplies the gate vector and the nonlinear hyperbolic tangent mapping result of the action unit gradient tensor after time-frequency domain projection transformation element by element, and then multiplies it by the weight value output by the dynamic decay function. The base of the decay function is 1.2 minus the sum of the absolute values ​​of the first norm of the action unit gradient tensor, which is 0.4 times. Frequency domain feature calibration is achieved by injecting the compensation amount output from the second channel into the residual connection point of the first channel. In the user intent probability vector generation stage, the temporal gradient of the compensated features is input into the long short-term memory network, and its output is multiplied by the exponential decay factor calculated based on the square of the second norm of the confused expression features to generate a six-dimensional intent probability distribution vector. At the same time, the temporal data stream of the change rate of the expression features is output synchronously, where the denominator of the decay factor contains the square value of the configurable inhibition coefficient. In the speech reconstruction stage, the denoised speech signal is reconstructed through the short-time Fourier inverse transform process: the phase angle information of the original audio features is preserved, and the compensated frequency domain features are multiplied element by element by the corresponding phase complex exponential function before performing the inverse spectral transform.

6. The AI-based eye health detection system according to claim 5, characterized in that: In the knowledge-enhanced semantic distillation module, the cross-modal embedding operation after the denoised speech signal is converted into text is specifically as follows: The system receives denoised speech signals and a medical symptom mapping dictionary as input. An end-to-end speech recognition model converts the denoised speech signals into word sequences, while simultaneously acquiring frame-level facial expression feature change rate data. A time alignment function maps the frame-level facial expression feature change rate into word-level facial expression feature vectors. A pre-trained language model in the medical field is used to process the word sequences to generate text embedding vectors. The text embedding vectors and word-level facial expression feature vectors are concatenated along the channel dimension to generate a joint embedding matrix that integrates text semantics and facial expression dynamic features. The time alignment operation uses a sliding window dynamic interpolation algorithm, calculating interpolation weights based on the offset between the speech recognition timestamp and the facial expression feature timestamp.

7. The AI-based eye health detection system according to claim 6, characterized in that: The specific operations of the structured extraction are as follows: A bidirectional long short-term memory network is constructed to process the fusion embedding vector sequence, generating forward and backward hidden state vectors respectively. The bidirectional hidden state vectors are concatenated and input into a fully connected layer to generate an emission score matrix. Entity transition constraints from a medical symptom mapping dictionary are introduced into the conditional random field decoding layer. The Viterbi algorithm is used to solve for the globally optimal entity label sequence, outputting a structured entity set containing symptom descriptions, entity types, and initial confidence levels.

8. The AI-based eye health detection system according to claim 7, characterized in that: The specific operation of dynamically correcting entity confidence based on the rate of change of facial expression features is as follows: Calculate the second norm of the acceleration of facial expression feature changes within the time period covered by each entity, and map it to an expression stability index through a trainable correlation matrix; compare the normalized difference between this index and the median stability index of the overall conversation, and convert the difference value into a confidence correction factor through an exponential decay function. The initial confidence level is multiplied by a correction factor to generate a calibrated confidence level, and entities below the threshold are filtered out. The specific operation of converting entity text into standard medical codes using a medical symptom mapping dictionary is as follows: apply the dictionary mapping function to each corrected entity to convert it into standard medical codes, and only retain entities with a confidence level greater than a set threshold. The specific operation of aggregating and generating standardized health indicator data streams by time window is as follows: Within a preset time window, entities of the type of symptoms are selected, each symptom entity is converted into a one-hot vector in the symptom coding space according to its standard medical code, and multiplied by the calibrated confidence of the entity. All weighted one-hot vectors within the window are summed to generate the standardized health indicator vector for that time window. Finally, the standardized health indicator vector sequence arranged in chronological order is output as the standardized health indicator data stream.

9. The AI-based eye health detection system according to claim 8, characterized in that: In the multimodal confidence circuit breaker module, the operation of detecting the temporal derivative difference between the speech description feature vector and the facial expression physiological feature vector specifically involves: Real-time acquisition of user intent probability vectors, standardized health indicator data streams, and temporal derivatives of facial physiological features as input data; First, semantic feature extraction is performed: the temporal derivatives of the three types of symptom subvectors (refractive, color vision, and visual field) in the standardized health indicator data stream are weighted and summed using the symptom-intent association matrix, and then activated by a normalized exponential function to generate semantic dynamic feature vectors. Next, perform the contradiction measurement operation: calculate the logarithmic ratio of the user intent probability vector to the semantic dynamic feature vector, superimpose a preset zero-prevention constant, and apply predefined masking rules to filter non-contradiction sensitive dimensions; input the filtered vector into the weighted singular value norm processor, and output the non-negative contradiction index value.

10. The AI-based eye health detection system according to claim 9, characterized in that: The specific steps for activating the re-questioning protocol and generating the eye health assessment report are as follows: Set a dynamic circuit breaker threshold parameter, which is generated by a linear combination of the base threshold and the absolute value of the second time derivative of the facial physiological feature vector; When the contradiction index exceeds the dynamic circuit breaker threshold and the confusion dimension component of the user intent probability vector is greater than 0.5, the reinforcement learning decision protocol is triggered: based on the current user intent state, contradiction index and facial expression physiological characteristics, a re-questioning strategy is selected from the action space and sent to the user. Otherwise, the standardized health indicator data stream is subjected to time dimension maximum pooling processing, which is then concatenated with the user intent probability vector to form a joint feature. After linear transformation by the weight matrix, the final eye health assessment report is generated.

Images