Deep learning based diopter detection data fusion processing system

Abstract

The application discloses a kind of based on deep learning diopter detection data fusion processing system, it is related to medical detection technical field.The system includes the multimodal data acquisition module, data preprocessing and alignment module, dynamic sequence analysis module, deep learning regression core module, individual calibration module and result output and visualization module connected in turn.The system obtains objective detection data by multimodal data acquisition module, forms multimodal data after preprocessing and space-time alignment;Dynamic sequence analysis module extracts adjustment response time sequence characteristics;Deep learning regression core module calculates initial diopter parameter;Individual calibration module combines user biological characteristics to optimize parameter;Finally, visual quality simulation result is generated through result output and visualization module.The application utilizes multimodal data fusion and deep learning technology, realizes the quick, accurate diopter automatic detection, significantly improves detection efficiency and individual level.

Description

Technical Field

[0001] This invention discloses a deep learning-based refractive error measurement data fusion and processing system, which relates to the field of medical testing technology. Background Technology

[0002] In the field of modern visual health testing, refractive error measurement is a key step in vision assessment; with the increasing demand for precision medicine, the limitations of traditional refractive error measurement methods are becoming increasingly apparent.

[0003] Currently, the mainstream testing methods mainly include two technical approaches: computerized refractometers and comprehensive refractometers. Computerized refractometers are based on infrared optics and estimate refractive power by measuring the pupil's reflectivity to light stimuli; comprehensive refractometers, on the other hand, rely on professional operation and determine the final prescription through trial lens combinations and the patient's subjective feedback.

[0004] However, traditional refractive error measurement systems face numerous challenges in practical applications: On the one hand, computerized refractometers calculate using a single optical parameter, making them susceptible to fluctuations in patient accommodation and changes in pupil state, resulting in insufficient stability of measurement results; on the other hand, while comprehensive refractometers can provide personalized prescriptions, the entire process heavily relies on the optometrist's experience, and the uncertainty of patient subjective feedback leads to low testing efficiency. More importantly, existing devices generally lack multi-dimensional data fusion capabilities, failing to effectively integrate the three-dimensional geometric features of the eyeball, dynamic accommodation response, and environmental parameters, making it difficult to achieve accurate personalized refractive error assessments when facing complex visual problems. Therefore, developing a refractive error measurement system that can integrate multimodal data, possess autonomous learning capabilities, and provide objective and accurate test results is of great significance. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, the present invention provides a deep learning-based refractive power measurement data fusion processing system, which solves the problems mentioned in the background art through the following scheme.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a deep learning-based refractive power measurement data fusion processing system, comprising:

[0007] A multimodal data acquisition module is used to simultaneously acquire objective detection data from multiple sensors;

[0008] The data preprocessing and alignment module is used to process the objective detection data to obtain multimodal data;

[0009] The dynamic sequence analysis module is used to process the multimodal data to obtain the timing characteristics of the regulation response;

[0010] The deep learning regression core module is used to process the temporal features of the adjustment response to obtain the initial diopter parameters;

[0011] A personalized calibration module is used to perform personalized calibration on the initial refractive power parameters to obtain optimized refractive power parameters.

[0012] The results output and visualization module is used to perform visualization rendering processing on the optimized refractive parameters and generate visual quality simulation results.

[0013] The deep learning-based refractive power measurement data fusion and processing system also includes a data quality control module, which is used to perform quality assessment processing on the objective detection data to obtain a data quality assessment signal.

[0014] The deep learning-based refractive power measurement data fusion and processing system also includes a process control module, which coordinates and controls the working status of each module and generates system workflow control signals.

[0015] The technical effects and advantages of this invention are as follows:

[0016] 1. This invention achieves synchronous acquisition and precise alignment of multi-source objective detection data such as infrared images, 3D point clouds, ambient light intensity, and dynamic optotypes through a multimodal data acquisition module and a data preprocessing and alignment module. This solves the problem of traditional optometry relying on a single optical parameter, greatly improves the data dimensionality and measurement objectivity, and lays a reliable data foundation for accurate refractive power calculation.

[0017] 2. This invention uses a hybrid neural network architecture to perform in-depth analysis of the temporal features of accommodative response through a dynamic sequence analysis module and a deep learning regression core module. It can accurately extract essential features related to refractive power from the dynamic response of the eyeball, effectively overcome the interference of patient accommodative fluctuations and subjective feedback uncertainties on the measurement, and significantly improve the accuracy and stability of the test results.

[0018] 3. This invention, through a personalized calibration module and a data quality control module, enables historical data traceability and parameter optimization based on user biometrics. At the same time, it ensures the reliability of collected data through real-time quality assessment and threshold judgment, realizing individualized adaptation and closed-loop quality control of the detection process, which greatly improves the practicality and clinical applicability of the system. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a system framework structure diagram.

[0021] Figure 2 This is a flowchart of the core data processing. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0023] See Figure 1 As shown, this invention provides an overall modular framework for a deep learning-based refractive power measurement data fusion and processing system. The system starts with a multimodal data acquisition module, sequentially connecting to a data preprocessing and alignment module, a dynamic sequence analysis module, a deep learning regression core module, a personalized calibration module, and a result output and visualization module, forming a complete data processing chain. A data quality control module is bidirectionally connected to the multimodal data acquisition module, enabling real-time monitoring and feedback control of data quality. A process control module is interconnected with all processing modules, responsible for coordinating the working status and process scheduling of each module.

[0024] A multimodal data acquisition module is used to simultaneously acquire objective detection data from multiple sensors;

[0025] This embodiment details the hardware configuration, synchronization mechanism, and data acquisition process of the module. The module integrates an infrared image acquisition unit, a 3D geometry acquisition unit, an environmental parameter acquisition unit, and a visual stimulus generation unit. All units are triggered by a unified clock signal, ensuring that the time synchronization accuracy of data acquisition is within 10ms. The infrared image acquisition unit uses an 850nm wavelength infrared light source to acquire raw pupil image data and raw corneal reflection point data at a frame rate of 30 frames per second, with a resolution of 640×480 pixels. For example, during acquisition, the pupil center coordinates are calculated using an image processing algorithm. Assuming the pupil area detected in the raw pupil image data is 200 pixels, this is converted to an actual pupil diameter of 4mm based on calibration parameters. The 3D geometry acquisition unit uses a depth sensor to acquire raw 3D point cloud data at a point cloud density of 100,000 points per second, with a depth accuracy of ±1mm. For example, the radius of curvature of the eyeball is calculated through point cloud registration. The environmental parameter acquisition unit monitors ambient light intensity in real time, with a range of 0-10000 lux and an accuracy of ±5%, for example, acquiring data under typical indoor lighting conditions of 500 lux. The visual stimulus generation unit generates dynamic targets on the display screen. The stimulus frequency is adjustable between 0.1-2Hz, and the movement range is ±30° of the visual field angle. For example, if the target is moved at a frequency of 1Hz and shifts 30° from its center position within 1 second, the displacement speed is calculated to be 30° / second. All sensor data is transmitted to the processing unit via a bus to ensure the integrity and synchronization of objective detection data.

[0026] Objective detection data includes raw pupil image data and raw corneal reflection point data acquired by the infrared image acquisition unit, raw three-dimensional point cloud data acquired by the three-dimensional geometry acquisition unit, ambient light intensity data acquired by the environmental parameter acquisition unit, and dynamic target position data generated by the visual stimulus generation unit.

[0027] This embodiment details the acquisition parameters, processing methods, and calculation examples for each data type. Raw pupil image data is acquired by an infrared camera at a resolution of 640×480 pixels and 30 frames per second, with features extracted through Gaussian filtering and contrast enhancement. Raw corneal reflective point data is acquired by the same unit, using a centroid localization algorithm to achieve 0.1 pixel accuracy, for example, calculating the coordinate displacement of reflective points from the image. Raw 3D point cloud data is acquired by a depth sensor at a density of 100,000 points per frame, with a registration error of less than 0.5 mm, used to reconstruct the 3D structure of the eye. Ambient light intensity data is acquired by a light sensor with a range of 0-10000 lux and an accuracy of ±5%, for example, recording illumination values ​​in real time during detection. Dynamic target position data is generated by software, recording the changes in target screen coordinates over time; the stimulus frequency is adjustable, for example, calculating the target's movement trajectory at a frequency of 2 Hz. All data is timestamped to form a multimodal dataset, ensuring consistency in subsequent processing.

[0028] The data preprocessing and alignment module is used to process the objective detection data to obtain multimodal data;

[0029] This embodiment details the data processing flow, algorithm implementation, and calculation example of this module. The module first performs image enhancement processing on the raw pupil image data acquired by the infrared image acquisition unit. A 5×5 pixel Gaussian filter kernel is used for noise reduction, with a standard deviation σ set to 1.5. Next, contrast enhancement with a gamma value of 1.2 is performed. For example, if a pixel in the original image has a grayscale value of 100, after gamma correction, it becomes 100. 1.2 ≈158. Simultaneously, centroid localization processing is performed on the original corneal reflectance data. Assuming the detected reflectance area contains 9 pixels, with coordinates (x...). i y i ), through formula

[0030] ,

[0031] The centroid coordinates x were calculated. c Achieving a positioning accuracy of 0.1 pixels. Point cloud registration processing is performed on the raw 3D point cloud data acquired by the 3D geometric acquisition unit. Using an iterative nearest-point algorithm, the registration error is less than 0.5mm after 10 iterations. For example, by aligning the point cloud coordinate system with the camera coordinate system through registration, the eyeball curvature radius is calculated to be 7.85mm. Finally, spatiotemporal alignment processing is performed, adding a unified timestamp to all data. At t=1.5s, the following data are simultaneously acquired: pupil diameter 4.2mm, corneal reflector coordinates (125.3, 98.7) pixels, corneal vertex coordinates (0, 0, 25) mm, ambient light intensity 350 lux, and normalized coordinates of dynamic target position (0.5, 0.3), forming spatiotemporally aligned multimodal data.

[0032] The multimodal data is obtained by spatiotemporally aligning the objective detection data, specifically including spatiotemporally aligned pupil image data, spatiotemporally aligned corneal reflection point data, spatiotemporally aligned three-dimensional point cloud data, spatiotemporally aligned ambient light intensity data, and spatiotemporally aligned dynamic target position data.

[0033] This embodiment details the calculation method and correlation of each type of aligned data. The spatiotemporally aligned pupil image data is obtained through image enhancement processing, for example, calculating the pupil diameter change from the initial 4.0 mm to 4.2 mm; the spatiotemporally aligned corneal reflector data is obtained through centroid localization, calculating the reflector point's movement from the initial position (120.5, 95.2) pixels to (125.3, 98.7) pixels, with a displacement vector of (4.8, 3.5) pixels; the spatiotemporally aligned 3D point cloud data is obtained through registration processing, calculating the eyeball surface curvature radius to be 7.85 mm; the spatiotemporally aligned ambient light intensity data retains the original value of 350 lux; the spatiotemporally aligned dynamic target position data is recorded as normalized coordinates (0.5, 0.3). All data are correlated using a timestamp t=1.5s, for example, based on the pupil diameter of 4.2 mm and the target position (0.5, 0.3) at this time, the accommodative response is calculated to be 1.5 diopters, forming a complete spatiotemporally aligned data frame.

[0034] See Figure 2 As shown, this invention demonstrates the key steps of the core data processing workflow. The workflow begins with multimodal data, which is then processed through dynamic sequence construction to form a dynamic stimulus-response sequence. Temporal feature extraction is then used to obtain the modulated response temporal features. A feature quality assessment node is included in the workflow; data with substandard feature quality is returned for reprocessing. Qualified features undergo feature encoding and regression calculation to obtain initial refractive power parameters. These parameters are then verified by a parameter validation node, ultimately generating a visual quality simulation result.

[0035] The dynamic sequence analysis module is used to process the multimodal data to obtain the timing characteristics of the regulation response;

[0036] This embodiment specifically illustrates the sequence construction method and feature extraction process of this module. The module first performs dynamic sequence construction processing on the spatiotemporally aligned multimodal data, selecting 50 consecutive data frames with a sampling rate of 5Hz within a 10-second time window. Each data frame contains the pupil diameter from the spatiotemporally aligned pupil image data, the coordinates of the reflection point from the spatiotemporally aligned corneal reflection point data, and the target distance from the spatiotemporally aligned dynamic target position data.

[0037] In a preferred embodiment, at t=1.5s, the accommodative demand corresponding to a target distance of 0.5m is 2.0D, the measured actual accommodative response is 1.5D, and the accommodative hysteresis is 0.5D; at t=1.7s, the target distance changes to 0.33m, the accommodative demand is 3.0D, the measured accommodative response is 2.4D, and the accommodative hysteresis is 0.6D, and so on to construct a complete dynamic stimulus-response sequence. Next, temporal feature extraction processing is performed on this dynamic stimulus-response sequence, calculating the pupil diameter change rate, for example, the pupil diameter changes from 4.2mm to 4.0mm in 0.2s, the change rate is -1.0mm / s; the displacement velocity of the corneal reflector point is calculated, for example, the reflector point moves from pixel coordinates (125.3, 98.7) to pixel (128.1, 96.5), the displacement velocity is:

[0038] pixels per second;

[0039] Simultaneously, statistical features such as the standard deviation of the regulation lag (0.12D) and the correlation coefficient of pupil diameter change (-0.85) were extracted, ultimately yielding a regulation response time-series feature vector containing 15 time-domain features and 8 frequency-domain features.

[0040] The deep learning regression core module is used to process the temporal features of the adjustment response to obtain the initial diopter parameters;

[0041] This embodiment details the neural network structure and regression calculation process of the module. The module first performs feature encoding on the 23-dimensional conditioning response temporal features. A fully connected layer maps the input features to a 256-dimensional high-dimensional feature representation. The weight matrix of this fully connected layer has a dimension of 23×256, and the bias vector has a dimension of 256. For example, the input feature vector x contains 23 feature values, such as the mean pupil diameter (4.1) and the standard deviation of conditioning hysteresis (0.12). After a linear transformation z=Wx+b, the high-dimensional feature representation is obtained through the ReLU activation function, where the output of the i-th neuron is... Assuming the input of the 50th neuron =1.2, then output =1.2, the input of the 100th neuron =-0.5, then output =0. Next, nonlinear regression calculation is performed on the high-dimensional feature representation. A fully connected neural network with two hidden layers is used. The first hidden layer maps the 256-dimensional input to 512-dimensional using the ReLU activation function. The second hidden layer maps the 512-dimensional input to 128-dimensional. Finally, the output layer maps the 128-dimensional input to 3-dimensional, corresponding to the three initial refractive power parameters: spherical power, cylindrical power, and astigmatic axis.

[0042] Preferably, after the high-dimensional feature representation is calculated by the first hidden layer, the weighted input of the 200th neuron is 2.8, and the output is 2.8 after ReLU activation; after the second hidden layer is calculated, the weighted input of the 80th neuron is 1.5, and the output is 1.5; the final output layer uses linear activation to calculate the spherical power -3.25D, the cylindrical power -1.75D, and the astigmatic axis 15°.

[0043] The deep learning regression core module processes the temporal features of the accommodation response to obtain the initial refractive power parameters, specifically through the following steps:

[0044] The time-series features of the adjustment response are first processed by feature encoding to obtain a high-dimensional feature representation;

[0045] The high-dimensional feature representation is then processed by nonlinear regression to obtain the initial diopter parameters.

[0046] This embodiment details the computational details and parameter examples for each layer of the neural network. In feature encoding processing, a 23×256 weight matrix W and a 256-dimensional bias vector b are used to encode the input feature vector. A linear transformation is performed, where x1=4.1, x2=0.12, ..., x23=0.78, to calculate the 256-dimensional intermediate feature z=Wx+b, which is then activated by ReLU to obtain the high-dimensional feature representation h.

[0047] In the nonlinear regression calculation, the first hidden layer uses a 256×512 weight matrix W1 and a 512-dimensional bias vector b1 to calculate the intermediate output h1=ReLU(W1h+b1).

[0048] The second hidden layer uses a 512×128 weight matrix W2 and a 128-dimensional bias vector b2 to calculate the intermediate output h2=ReLU(W2h1+b2).

[0049] The output layer uses a 128×3 weight matrix W3 and a 3-dimensional bias vector b3 to calculate the initial refractive power parameter y=W3h2+b3, resulting in a spherical power of -3.25D, a cylindrical power of -1.75D, and an astigmatic axis of 15°, thus completing the end-to-end mapping from the adjustment response timing characteristics to the initial refractive power parameter.

[0050] A personalized calibration module is used to perform personalized calibration on the initial refractive power parameters to obtain optimized refractive power parameters.

[0051] This embodiment details the personalized recognition and parameter optimization process of the module. The module first performs iris texture feature extraction on the enhanced pupil image data, dividing the 640×480 pixel iris region into eight concentric ring sectors. 256-dimensional Gabor texture features are extracted from each sector, and principal component analysis is used to reduce the dimensionality to a 32-dimensional feature vector. Finally, a hash function is used to generate a 64-bit user biometric identifier.

[0052] Preferably, the iris image of the current user is processed to obtain a feature vector [0.12, -0.05, 0.33, ..., 0.21], which is then hashed to generate a unique identifier 0x9A7F3E2D1C. Subsequently, historical data retrieval is performed based on the user's biometric identifier. The most recent 10 detection records corresponding to this identifier are queried in the user's historical database to obtain the historical spherical power sequence [-3.20, -3.18, -3.22, -3.25, -3.15, -3.28, -3.23, -3.19, -3.26, -3.21]D;

[0053] Historical cylindrical power sequence [-1.70, -1.72, -1.68, -1.75, -1.65, -1.78, -1.73, -1.69, -1.76, -1.71]D;

[0054] Historical astigmatism axis sequence [16, 14, 17, 15, 13, 18, 16, 14, 17, 15]°.

[0055] Finally, the initial refractive power parameters and the user's historical refractive power data are optimized using a weighted fusion algorithm. The current measurement weight is set to 0.7, and the historical data weight is set to 0.3. The spherical power is calculated as -3.25×0.7+(-3.21)×0.3=-3.24D, the cylindrical power is calculated as -1.75×0.7+(-1.71)×0.3=-1.74D, and the astigmatic axis is calculated as 15×0.7+15×0.3=15°, thus obtaining the optimized refractive power parameters.

[0056] The personalized calibration module performs personalized calibration on the initial refractive power parameters to obtain optimized refractive power parameters, which is achieved through the following steps:

[0057] First, iris texture feature extraction is performed on the pupil image data to obtain the user's biometric identifier;

[0058] Subsequently, historical data retrieval and processing are performed based on the user's biometric identifier to obtain the user's historical refractive error data;

[0059] Finally, the initial refractive error parameters and the user's historical refractive error data are optimized to obtain the optimized refractive error parameters.

[0060] This embodiment details the algorithm implementation and numerical calculation process for each step. In the iris texture feature extraction process, the iris region in the enhanced pupil image data is normalized, and the circular iris is unfolded into a 64×512 pixel rectangular image. A 2D Gabor filter bank is used to extract texture phase information, generating a 2048-bit original feature code. After dimensionality reduction and hash mapping, a 64-bit user biometric identifier is obtained. In the historical data retrieval process, a query operation is performed in the SQL database based on the user's biometric identifier to retrieve 10 valid detection records for the user within the last 3 months. The mean of the historical spherical power is -3.21D, and the standard deviation is 0.04D; the mean of the historical cylindrical power is -1.71D, and the standard deviation is 0.04D; and the mean of the historical astigmatism axis is 15°, and the standard deviation is 1.5°. In the parameter optimization process, based on the Bayesian estimation principle, the initial refractive power parameter is used as the new observation value, and the user's historical refractive power data is used as the prior distribution. Data fusion is performed using Kalman filtering. The specific calculation formula is as follows:

[0061] ,

[0062] The final optimized refractive power parameters are: spherical power -3.24D, cylindrical power -1.74D, and astigmatism axis 15°, completing the personalized calibration process.

[0063] The results output and visualization module is used to perform visualization rendering processing on the optimized refractive parameters and generate visual quality simulation results.

[0064] This embodiment details the quality assessment and image generation process of the module. The module first performs confidence calculations on the optimized refractive error parameters, calculating a confidence score based on the consistency between the current measurement and the user's historical refractive error data.

[0065] Preferably, the spherical power of -3.24D differs from the historical mean of -3.21D by 0.03D, which is within the historical standard deviation of 0.04D, and the spherical power confidence score is 0.85.

[0066] The cylinder power of -1.74D differs from the historical mean of -1.71D by 0.03D, which is within the historical standard deviation of 0.04D, and the cylinder confidence score is 0.82; the astigmatism axis of 15° is the same as the historical mean of 15°, and the axis position confidence score is 0.95.

[0067] The overall confidence score is (0.85+0.82+0.95) / 3=0.87, generating the quality assessment result.

[0068] Subsequently, the optimized refractive power parameters and the quality assessment results are combined for visualization rendering. The point spread function is used to simulate optical distortion. The Gaussian blur kernel size σ = 3.24 pixels is calculated based on the spherical power of -3.24D. The anisotropic blur parameters are calculated based on the cylindrical power of -1.74D and the axis of 15°. The visual quality simulation results are generated on a 1280×720 pixel canvas, where the quality assessment watermark is displayed with a confidence level of 0.87 and 87% transparency.

[0069] The results output and visualization module performs visualization rendering on the optimized refractive parameters to generate visual quality simulation results. Specifically, this is achieved through the following steps: First, the confidence level of the optimized refractive parameters is calculated to obtain the quality assessment result; then, the optimized refractive parameters and the quality assessment result are combined for visualization rendering to generate the visual quality simulation result.

[0070] This embodiment details the calculation process of the optical simulation. In the confidence score calculation, a confidence score algorithm based on historical consistency is used to calculate the Z-score statistics of the current spherical power -3.24D and the historical sequence [-3.20, -3.18, -3.22, -3.25, -3.15, -3.28, -3.23, -3.19, -3.26, -3.21]D. The confidence level of the spherical mirror is obtained by mapping using the sigmoid function. ;

[0071] The same method was used to calculate the cylinder confidence score of 0.82 and the axis position confidence score of 0.95. The final quality assessment result includes an overall confidence score of 0.87 and confidence scores for each component. In the visualization rendering process, a point spread function is constructed based on the physical optics model. ,in:

[0072] =8.8 pixels,

[0073] =11.1 pixels;

[0074] The standard test image is convolved with PSF to generate a blurred simulated image, and a confidence watermark with 87% transparency is superimposed. The final output is a visual quality simulation result containing optimized refractive power parameters -3.24D / -1.74D×15° and a quality assessment result of 0.87.

[0075] The deep learning-based refractive power measurement data fusion and processing system also includes a data quality control module, which is used to perform quality assessment processing on the objective detection data to obtain a data quality assessment signal.

[0076] This embodiment details the quality assessment algorithm and signal generation process of the data quality control module. The data quality control module receives objective detection data in real time from the multimodal data acquisition module, including raw pupil image data, raw corneal reflective point data, raw 3D point cloud data, ambient light intensity data, and dynamic target position data. Image sharpness is assessed on the raw pupil image data by calculating the sum of image gradient magnitudes. For example, in a 640×480 pixel image, the gradient value of each pixel is calculated using the following formula:

[0077] ,

[0078] in and These represent the Sobel operator convolution results in the horizontal and vertical directions, respectively. When the sharpness score is lower than the preset threshold of 0.7, it is marked as a quality anomaly.

[0079] The original corneal reflection point data is evaluated for reflection intensity. The average gray value of the reflection point area is required to be no less than 180 (8-bit gray value range 0-255). If the average gray value of the reflection point in a certain frame is detected to be 150, the quality of the frame is deemed unqualified.

[0080] The integrity of the original 3D point cloud data is assessed by calculating the ratio of valid points to theoretical points. The ratio should be no less than 90%. For example, if there are 85,000 valid points out of 100,000 point cloud data points, the integrity score is 0.85.

[0081] The stability of the ambient light intensity data is evaluated by calculating the coefficient of variation of illuminance within 10 seconds. The coefficient of variation should not exceed 0.1. For example, the standard deviation of the illuminance sequence [500, 520, 480, 510, 490 lux] is 15.8, the mean is 500, and the coefficient of variation is 0.032, which meets the requirements.

[0082] The trajectory conformity of the dynamic target position data is evaluated, and the root mean square error between the actual target position and the theoretical position is calculated, which is required to be no more than 2 pixels.

[0083] Finally, the data quality assessment signal Q is calculated by weighting the scores of each component: Q = 0.3 × image sharpness score + 0.3 × reflection intensity score + 0.2 × point cloud integrity score + 0.1 × illumination stability score + 0.1 × trajectory conformity score. When Q ≥ 0.8, a high-quality signal is output; when 0.6 ≤ Q < 0.8, a medium-quality signal is output; and when Q < 0.6, a low-quality signal is output.

[0084] The data quality control module also includes a quality judgment unit, which performs threshold judgment processing on the data quality assessment signal to generate a data re-acquisition control signal; the multimodal data acquisition module re-acquisitions objective detection data according to the data re-acquisition control signal.

[0085] This embodiment details the judgment logic and re-acquisition control mechanism of the quality judgment unit. The quality judgment unit receives the data quality evaluation signal Q and sets three judgment thresholds: a high-quality threshold and a low-quality threshold. Qualified threshold Re-collection threshold .

[0086] when When this happens, a "high-quality" signal is generated, allowing the process to proceed to the next stage.

[0087] when When the time is right, a "qualified" signal is generated, the system records a quality warning but continues processing;

[0088] when At that time, a data reacquisition control signal is generated.

[0089] For example, if a data quality assessment signal Q=0.55 is obtained in a certain test, which is lower than the reacquisition threshold of 0.6, the quality judgment unit immediately generates a data reacquisition control signal. This signal contains the reacquisition instruction, the type of quality defect (such as insufficient point cloud integrity), and suggested adjustment parameters (such as adjusting the acquisition distance).

[0090] After receiving the data reacquisition control signal, the multimodal data acquisition module reinitializes each sensor within 0.5 seconds. Based on the suggested parameters in the signal, it adjusts the infrared camera gain, depth sensor sampling rate, and target motion parameters, and reacquires 15 seconds of objective detection data. During the reacquisition process, the system automatically eliminates factors that could cause quality defects, such as prompting the user to keep their head stable and adjusting ambient lighting, ensuring that the reacquired data quality assessment signal Q is improved to above 0.75. Only after meeting the pass standard can the data proceed to the data preprocessing and alignment module.

[0091] The deep learning-based refractive power measurement data fusion and processing system also includes a process control module, which coordinates and controls the working status of each module and generates system workflow control signals.

[0092] This embodiment details the state coordination mechanism and process control signal generation process of the process control module. The process control module monitors the working status of each module in real time through a state detection unit, collecting state data every 100 milliseconds. For the multimodal data acquisition module, it monitors the sensor readiness status and data acquisition progress. If the infrared image acquisition unit, 3D geometry acquisition unit, environmental parameter acquisition unit, and visual stimulus generation unit are all ready (each with a readiness flag of 1), and the data acquisition progress reaches 100%, an acquisition completion signal is generated. For the data preprocessing and alignment module, it monitors the processing queue length and computational load. When the number of data frames to be processed in the processing queue exceeds 50 frames or the CPU utilization exceeds 85% for 3 consecutive seconds, a load alarm signal is generated. For the dynamic sequence analysis module, it monitors the feature extraction progress. When the number of feature extractions completed for 50 data frames reaches 50, a sequence analysis completion signal is generated.

[0093] The process control module performs coordinated control processing based on the status monitoring results, using a finite state machine to implement process scheduling. After system initialization, it first sends a start signal to the multimodal data acquisition module and simultaneously starts a 15-second countdown. When it receives the acquisition completion signal and the countdown ends, it sends a processing start signal to the data preprocessing and alignment module. After completing the spatiotemporal alignment processing of all 50 frames of data, the data preprocessing and alignment module returns a completion status code 200 to the process control module, which then sends a sequence construction instruction to the dynamic sequence analysis module. Upon receiving the instruction, the dynamic sequence analysis module begins constructing a dynamic stimulus-response sequence. When the 23-dimensional conditioning response temporal features are calculated, it returns a status code 200, triggering the process control module to send a regression calculation instruction to the deep learning regression core module. After the deep learning regression core module completes the initial refractive error parameter calculation, it returns a status code 200. The process control module immediately sends a calibration instruction to the personalized calibration module. After the personalized calibration module completes parameter optimization based on the user's biometric identifier, it returns a status code 200. Finally, the process control module sends a rendering instruction to the result output and visualization module, completing the entire processing flow.

[0094] During the coordinated control process, the process control module generates system workflow control signals in real time. These signals include the current process stage identifier, status codes for each module, timestamps, and quality control flags. For example, at t=15.2 seconds, the system is in the dynamic sequence analysis stage. The process control signals include the stage identifier "Sequence Analysis," status code 200 for the multimodal data acquisition module, status code 200 for the data preprocessing and alignment module, status code 101 (processing) for the dynamic sequence analysis module, status code 100 (waiting) for the deep learning regression core module, status code 100 (waiting) for the personalized calibration module, status code 100 (waiting) for the result output and visualization module, and the quality control flag "0" (normal). When any module returns an abnormal status code (e.g., status code 500 from the data preprocessing and alignment module indicates processing failure), the process control module immediately generates an exception handling signal, interrupts the current process, and reinitializes the relevant modules to ensure the reliability and continuity of system operation. Through this refined coordinated control mechanism, the system achieves seamless connection and efficient collaborative work between various processing modules.

[0095] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other.

[0096] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A deep learning-based refractive power measurement data fusion and processing system, characterized in that, The system includes: A multimodal data acquisition module is used to simultaneously acquire objective detection data from multiple sensors; The data preprocessing and alignment module is used to process the objective detection data to obtain multimodal data; The objective detection data includes raw pupil image data and raw corneal reflection point data acquired by the infrared image acquisition unit, raw three-dimensional point cloud data acquired by the three-dimensional geometry acquisition unit, ambient light intensity data acquired by the environmental parameter acquisition unit, and dynamic target position data generated by the visual stimulus generation unit. The dynamic sequence analysis module is used to process the multimodal data to obtain the timing characteristics of the regulation response; The deep learning regression core module is used to process the temporal features of the adjustment response to obtain the initial diopter parameters; The deep learning regression core module processes the temporal features of the accommodation response to obtain the initial refractive power parameters, specifically through the following steps: The time-series features of the adjustment response are first processed by feature encoding to obtain a high-dimensional feature representation; The high-dimensional feature representation is then processed by nonlinear regression to obtain the initial diopter parameters; A personalized calibration module is used to perform personalized calibration on the initial refractive power parameters to obtain optimized refractive power parameters. The personalized calibration module performs personalized calibration on the initial refractive power parameters to obtain optimized refractive power parameters, specifically through the following steps: First, iris texture feature extraction is performed on the pupil image data to obtain the user's biometric identifier; Subsequently, historical data retrieval and processing are performed based on the user's biometric identifier to obtain the user's historical refractive error data; Finally, the initial refractive error parameters and the user's historical refractive error data are optimized to obtain the optimized refractive error parameters. The results output and visualization module is used to perform visualization rendering processing on the optimized refractive parameters and generate visual quality simulation results.

2. The deep learning-based refractive power measurement data fusion processing system according to claim 1, characterized in that, The multimodal data is obtained by spatiotemporally aligning the objective detection data, specifically including spatiotemporally aligned pupil image data, spatiotemporally aligned corneal reflection point data, spatiotemporally aligned three-dimensional point cloud data, spatiotemporally aligned ambient light intensity data, and spatiotemporally aligned dynamic target position data.

3. The deep learning-based refractive power measurement data fusion processing system according to claim 1, characterized in that, The modulated response temporal features are obtained through the following steps: the multimodal data undergoes dynamic sequence construction processing to obtain a dynamic stimulus-response sequence; the dynamic stimulus-response sequence then undergoes temporal feature extraction processing to obtain the modulated response temporal features.

4. The deep learning-based refractive power measurement data fusion processing system according to claim 1, characterized in that, The result output and visualization module performs visualization rendering on the optimized refractive parameters to generate visual quality simulation results, which is achieved through the following steps: First, the confidence level of the optimized refractive parameters is calculated to obtain the quality assessment results. Subsequently, the optimized refractive power parameters and the quality assessment results are combined for visualization rendering to generate the visual quality simulation results.

5. The deep learning-based refractive power measurement data fusion processing system according to claim 1, characterized in that, The system also includes a data quality control module, which is used to perform quality assessment processing on the objective detection data to obtain a data quality assessment signal.

6. The deep learning-based refractive power measurement data fusion processing system according to claim 5, characterized in that, The data quality control module further includes a quality judgment unit, which is used to perform threshold judgment processing on the data quality assessment signal and generate a data re-acquisition control signal; the multimodal data acquisition module re-acquisitions objective detection data according to the data re-acquisition control signal.

7. The deep learning-based refractive power measurement data fusion processing system according to claim 1, characterized in that, The system also includes a process control module, which is used to coordinate and control the working status of each module and generate system workflow control signals.

Images