A visual aid device virtual pre-evaluation method and system based on spatiotemporal pathological characteristics

Abstract

This invention discloses a virtual pre-evaluation method and system for visual assistive devices based on spatiotemporal pathological features. The method includes: acquiring pathological parameters and constructing a virtual scene; generating a sensitivity mask, frequency band loss coefficients, and temporal contrast sensitivity model parameters; performing field-of-view cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling on the input image when the visual assistive device model is enabled to obtain the image to be processed; performing spatial frequency degradation processing, temporal contrast modulation processing, and visual defect area perceptual filling processing on the image to be processed to obtain the target output image; and collecting the task execution results of the target output image in a standardized virtual task and outputting the evaluation results. This invention can quantitatively pre-evaluate the assistive effect of visual assistive devices without a physical prototype.

Description

Technical Field

[0001] This invention relates to the fields of computer vision simulation, virtual reality interaction, image processing and ophthalmic assistive devices, and in particular to a virtual pre-evaluation method and system for visual assistive devices based on spatiotemporal pathological features. Background Technology

[0002] Existing visual simulation solutions mostly focus on single-dimensional processing such as blurred vision, visual field obstruction, or simple contrast changes, making it difficult to simultaneously reflect the complex abnormalities in spatial frequency perception, temporal frequency perception, and perceptual filling of visual field defects in patients with eye diseases. Furthermore, for the evaluation of the effectiveness of visual assistive devices, existing solutions typically process image enhancement and pathological visual perception separately, lacking a closed-loop evaluation mechanism that couples the parametric model of the visual assistive device, the pathological visual perception model, and the standardized virtual task scenario. Therefore, it is difficult to quantitatively compare the assistive effects of different assistive device parameters on different pathological states without physical prototypes. Based on this, there is an urgent need for a virtual pre-evaluation method and system that can jointly simulate the processing chain of visual assistive devices and the pathological visual perception processing chain in a virtual reality scenario and output objective evaluation results. Summary of the Invention

[0003] To address the aforementioned issues, this invention provides a virtual pre-evaluation method and system for visual assistive devices based on spatiotemporal pathological features. This method constructs a virtual scene within a virtual reality environment, establishes a visual assistive device model incorporating field-of-view cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling, and establishes a pathological visual perception model incorporating spatial frequency degradation processing, temporal contrast modulation processing, and visual defect area perceptual filling processing. This allows for objective evaluation of task execution results under the same standardized virtual task configuration, both with and without the visual assistive device model enabled, achieving a quantitative pre-evaluation of the assistive effect of specific device parameter schemes under specific pathological conditions.

[0004] The first objective of this invention is to provide a virtual pre-assessment method for visual aids based on spatiotemporal pathological features; The technical solution provided by this invention is as follows: A virtual pre-assessment method for visual aids based on spatiotemporal pathological features includes the following steps: The pathological parameters of the object to be evaluated are obtained, and a virtual scene is constructed in a virtual reality environment based on the pathological parameters to generate a sensitivity mask, frequency band loss coefficient and time contrast sensitivity model parameters corresponding to the pathological parameters. When the visual assistive device model is enabled, the constructed visual assistive device model sequentially performs field cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling on the input image of the virtual scene to obtain the image to be processed. The image to be processed is subjected to pathological visual perception processing to obtain the target output image; The target output image is subjected to a virtual task test, and the task execution results are collected to output an evaluation result of the visual aid device's assistive effect.

[0005] Preferably, the step of sequentially performing field-of-view cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling on the input image of the virtual scene using the constructed visual assistive device model to obtain the image to be processed specifically includes: Perform field-of-view cropping and resampling on the input image; Gamma correction is performed on the cropped and resampled image to enable dynamic range mapping; Perform CLAHE local contrast enhancement on the luminance channel of the dynamically range-mapped image; Edge enhancement is achieved by using the Laplacian operator or Gaussian smoothing difference to sharpen the image after local contrast enhancement. The image with enhanced edges is modeled for system latency in milliseconds by using a circular buffer to cache historical frames; and the output update is limited to a preset refresh rate, and samples are taken at the refresh time and displayed with frame hold to model the refresh rate limit in order to obtain the image to be processed.

[0006] Preferably, the step of performing pathological visual perception processing on the image to be processed to obtain the target output image specifically includes: Based on the sensitivity mask and the frequency band loss coefficient, multi-channel Mipmap sampling is performed on the image to be processed to construct low-frequency, mid-frequency and high-frequency channels, and the spatial frequency degraded visual image is reconstructed according to preset or dynamically determined weights. The retinal slip angular velocity and temporal frequency are calculated based on key information of objects in the scene. The pathological weights are generated by combining the retinal slip angular velocity and temporal frequency with the parameters of the temporal contrast sensitivity model. The pathological weights are then used to modulate temporal integration, contrast compression, and saliency attenuation. For visually defective regions, low-frequency texture sampling is performed based on the defect boundary and smoothed sampling coordinates to obtain the target visual fill color, and the target visual fill color is written into the frame buffer to obtain the target output image.

[0007] Preferably, the step of performing multi-channel Mipmap sampling on the image to be processed based on the sensitivity mask and the frequency band loss coefficient, constructing low-frequency, mid-frequency, and high-frequency channels, and reconstructing the spatially frequency-degraded visual image according to preset or dynamically determined weights, specifically includes: The sensitivity mask is sampled based on the texture coordinates of the current pixel to obtain the local visual sensitivity; The target blurring degree of the low-frequency channel, mid-frequency channel and high-frequency channel is calculated based on the local visual sensitivity and the frequency band loss coefficient, respectively. The scene texture is sampled three times in parallel according to the target blur level to obtain the color components of the low-frequency channel, the mid-frequency channel and the high-frequency channel; The visual image is obtained by linearly superimposing the color components and preset or dynamically determined weights.

[0008] Preferably, the preset or dynamically determined weights are jointly determined by local visual sensitivity and low-frequency loss coefficient, mid-frequency loss coefficient and high-frequency loss coefficient; and the target ambiguity corresponding to the low-frequency channel, mid-frequency channel and high-frequency channel satisfies the preset order constraint to maintain the stability and monotonicity of the frequency band construction.

[0009] Preferably, the step of calculating the retinal slip angular velocity and temporal frequency based on key information of scene objects, generating pathological weights by combining the retinal slip angular velocity and temporal frequency with the parameters of the temporal contrast sensitivity model, and using the pathological weights to modulate temporal integration, contrast compression, and significance attenuation, specifically includes: Input the world coordinate velocity, distance and geometric boundary information of objects in the scene, the user's gaze direction, the observer, the camera pose information and preset parameter data; The world coordinate velocity is combined with the observer or camera pose information and mapped to the retinal coordinate system to obtain the projected angular velocity. The time frequency is calculated based on the projected angular velocity and the target's local spatial frequency characteristics; The time sensitivity ratio is calculated based on the individualized damage coefficient and the time-comparison sensitivity model. The time sensitivity ratio is subjected to nonlinear gating mapping to generate normalized pathological weights; The time integration, contrast compression, and significance attenuation are modulated according to the pathological weights; The key information includes: world coordinate velocity, distance and geometric boundary information, user's gaze direction, observer, camera pose information, and preset parameter data.

[0010] Preferably, the step of performing low-frequency texture sampling based on the defect boundary and smoothed sampling coordinates for the visually defective region to obtain the target visual fill color, and writing the target visual fill color into the frame buffer to obtain the target output image, specifically includes: Define the visual defect region and calculate the defect boundary based on the defect mask; Boundary anchor points are determined based on the defective boundary, and smoothed sampling coordinates are calculated based on a nonlinear weighting function; Low-frequency texture sampling is performed based on the smoothed sampling coordinates to obtain the target visual fill color of the defective area; The target visual fill color is written to the frame buffer to obtain the target output image.

[0011] Preferably, the visual defect area is generated by mapping the clinical visual field sensitivity distribution to visual field angle coordinates from screen pixels and thresholding, and the temporal and nasal directions are corrected according to the relationship between the left and right eyeglass images.

[0012] Preferably, the pathological parameters include at least clinical visual field sensitivity distribution parameters, contrast sensitivity related parameters, and time-contrast sensitivity related parameters, wherein: The clinical visual field sensitivity distribution parameters are used to generate the sensitivity mask or the visual defect region; The contrast sensitivity related parameters are used to generate low-frequency loss coefficients, mid-frequency loss coefficients, and high-frequency loss coefficients; The time-comparison sensitivity parameters are used to generate individualized damage coefficients or time-comparison sensitivity model parameters.

[0013] The second objective of this invention is to provide a virtual pre-assessment system for visual aids based on spatiotemporal pathological features; The technical solution provided by this invention is as follows: A virtual pre-assessment system for visual aids based on spatiotemporal pathological features, comprising: The parameter generation and scene construction module is used to obtain the pathological parameters of the object to be evaluated, and construct a virtual scene in a virtual reality environment based on the pathological parameters to generate sensitivity mask, frequency band loss coefficient and time contrast sensitivity model parameters corresponding to the pathological parameters. The visual assistive device simulation module is used to sequentially perform field cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling on the input image of the virtual scene through the constructed visual assistive device model when the visual assistive device model is enabled, so as to obtain the image to be processed. The pathological visual simulation module is used to perform pathological visual perception processing on the image to be processed in order to obtain the target output image. The evaluation module is used to perform virtual task tests on the target output image and collect the task execution results to output the evaluation results of the visual aid device's assistive effect.

[0014] Compared with existing technologies, this invention provides a virtual pre-evaluation method for visual assistive devices based on spatiotemporal pathological features, comprising the following steps: acquiring pathological parameters of the object to be evaluated, and constructing a virtual scene in a virtual reality environment based on the pathological parameters to generate sensitivity masks, frequency band loss coefficients, and temporal contrast sensitivity model parameters corresponding to the pathological parameters; when the visual assistive device model is enabled, sequentially performing field cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling on the input image of the virtual scene through the constructed visual assistive device model to obtain an image to be processed; performing pathological visual perception processing on the image to be processed to obtain a target output image; performing virtual task testing on the target output image and collecting the task execution results to output an evaluation result of the assistive effect of the visual assistive device; this invention couples the visual assistive device processing chain with the pathological visual perception processing chain in a unified virtual reality scene, and realizes the virtual pre-evaluation of the assistive effect of the visual assistive device through the behavioral results under the enabled visual assistive device model state. This scheme can not only simulate complex pathological features such as spatial frequency selective loss, temporal domain perception anomaly, and perceptual filling of visual defect areas, but also quantitatively compare the auxiliary effects of different equipment parameter schemes without physical prototypes, thereby improving the efficiency of design screening and scheme verification.

[0015] The present invention also provides a virtual pre-evaluation system for visual aids based on spatiotemporal pathological features. Since this system and the virtual pre-evaluation method for visual aids based on spatiotemporal pathological features solve the same technical problem and belong to the same technical concept, they should have the same beneficial effects, and will not be described in detail here. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a virtual pre-assessment method for visual aids based on spatiotemporal pathological features, provided as one embodiment; Figure 2 A flowchart of a virtual pre-assessment method for visual aids based on spatiotemporal pathological features, provided for another embodiment; Figure 3A diagram illustrating the effect of device preprocessing on a dynamic EVA example image, based on a visual aid device model provided in one embodiment. Figure 4 A diagram illustrating the effect of spatial frequency degradation processing based on clinical visual field sensitivity distribution in one embodiment; Figure 5 Comparison diagram of temporal contrast modulation processing based on retinal slip and eccentric angle gating provided in one embodiment; Figure 6 A comparison image of visual defect area perceptual filling processing provided in one embodiment and traditional black spot occlusion and low-pass blur filling; Figure 7 A visual behavior data feedback graph provided in one embodiment; Figure 8 A structural diagram of a virtual pre-assessment system for visual aids based on spatiotemporal pathological features, provided in one embodiment; Figure 9 This is a schematic diagram of the structure of an electronic device provided in one embodiment. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] like Figures 1 to 2 As shown, this embodiment of the invention provides a virtual pre-evaluation method for visual assistive devices based on spatiotemporal pathological features, comprising the following steps: S1. Obtain the pathological parameters of the object to be evaluated, and construct a virtual scene in a virtual reality environment based on the pathological parameters to generate a sensitivity mask, frequency band loss coefficient and time contrast sensitivity model parameters corresponding to the pathological parameters; S2. When the visual assistive device model is enabled, the input image of the virtual scene is sequentially subjected to field cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement and temporal resampling through the constructed visual assistive device model to obtain the image to be processed; S3. Perform pathological visual perception processing on the image to be processed to obtain the target output image; S4. Perform a virtual task test on the target output image and collect the task execution results to output the evaluation results of the visual aid device's assistive effect.

[0020] In the above steps, step S1 is used to establish the mapping relationship between individualized pathological parameters and virtual scenes; step S2 is used to acquire the image to be processed of the visual assistive device model in the enabled state; step S3 is used to perform unified pathological visual perception processing on the image to be processed; step S4 is used to acquire the task execution result of the target output image in the standardized virtual task, and output the evaluation result of the assistive effect of the visual assistive device according to at least one of the following: task completion time, number of collisions, gait parameters, head movement parameters, hand movement parameters, eye movement gaze heatmap, gaze duration and saccade time, thereby realizing the virtual pre-evaluation of the assistive effect of the visual assistive device under the condition of no physical prototype.

[0021] In step S1, a physically consistent 3D virtual scene is built in the Unity platform. Combined with a parameterizable environmental element model, including spatial structure, material properties, lighting conditions and dynamic objects, a daily life scenario is digitally reconstructed. Under a unified rendering and interaction framework, the scene state is linked with the user's perspective and behavioral logic for simulation. Thus, repeatable and controllable virtual environment experiments and verifications can be achieved without the need for real venues and physical construction.

[0022] The pathological parameters include at least clinical visual field sensitivity distribution parameters, contrast sensitivity related parameters, and time contrast sensitivity related parameters, wherein: the clinical visual field sensitivity distribution parameters are used to generate sensitivity masks or visual defect regions; the contrast sensitivity related parameters are used to generate low-frequency loss coefficients, mid-frequency loss coefficients, and high-frequency loss coefficients; and the time contrast sensitivity related parameters are used to generate individualized damage coefficients or time contrast sensitivity model parameters.

[0023] Preferably, the step of sequentially performing field-of-view cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling on the input image of the virtual scene using the constructed visual assistive device model to obtain the image to be processed specifically includes: A1. Perform field-of-view cropping and resampling on the input image; A2. Perform Gamma correction on the cropped and resampled image to perform dynamic range mapping; A3. Perform CLAHE local contrast enhancement on the luminance channel of the dynamically range-mapped image; A4. Enhance the edges of the locally contrast-enhanced image by using the Laplacian operator or Gaussian smoothing difference inverse masking sharpening; A5. The edge-enhanced image is modeled with system latency in milliseconds by caching historical frames using a circular buffer; and the output update is limited to a preset refresh rate, sampling is performed at the refresh time and frame hold display is used to perform refresh rate limitation modeling to obtain the image to be processed.

[0024] In practical applications, a parametric model of a visual assistive device is constructed based on local contrast enhancement operators, edge enhancement operators, and temporal resampling parameters. Preferably, the visual assistive device is an electronic visual aid (EVA). The local contrast enhancement operator is preferably implemented using the CLAHE family of algorithms, the edge enhancement operator is preferably implemented using the Laplacian operator or its equivalent high-pass enhancement, and the temporal resampling module is used to simulate the device system's latency and refresh rate limitations. Therefore, this parametric model can improve the local contrast, edge sharpness, and detail discernibility of the image while magnifying the display, and reflects the impact of the device's physical latency and refresh rate limitations on dynamic images. The implementation effect of this visual assistive device model is as follows: Figure 3 As shown. Figure 3 The device preprocessing effect is demonstrated using a continuous lateral translation sequence of dynamic EVA example images. These example images consist of circles, rectangles, line segments, checkerboard patterns, and stripes of different spatial frequencies at varying scales, allowing simultaneous observation of changes in geometric contours, local contrast, and spatial frequency details throughout the device processing chain. The core parameters set in this embodiment are: digital magnification of 1.5x and system latency of 120 milliseconds. Figure 3 (a) to Figure 3 (d) shows the input image sequence at different time points. The dashed box in the figure indicates the input area corresponding to the device's zoom-in and cropping. Figure 3 (e) to Figure 3 (h) represents the device preprocessed output image obtained at the corresponding time point after field-of-view cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling, respectively corresponding to... Figure 3 (a) to Figure 3 The cropping input area is shown in the yellow dashed box in (d).

[0025] The above tests show that the algorithm achieves a 1.5x field-of-view magnification through field-of-view cropping and resampling. Under the combined processing of dynamic range mapping, CLAHE local contrast enhancement, and edge enhancement, low-contrast details that were difficult to discern in the original input image become clearer in the assisted preprocessed image, thus simulating the improvement in local contrast and detail recognition by visual aids. Simultaneously, temporal resampling implemented through a circular buffer can simulate the physical time delay limitations of visual aids. At the same timestamp, moving targets in the assisted preprocessed image exhibit a positional lag relative to the original input image, reflecting motion discontinuities and ghosting caused by device latency.

[0026] In step A1, the output field of view is calculated by performing image cropping and resampling on the image in the virtual environment. The geometric transformation includes: (1) According to the digital magnification Determine the size of the cropping window so that its width and height are respectively the width and height of the input image. ; (2) Cropping is performed using the image center point or a preset gaze point as anchor points; (3) Resample the cropped image to the display resolution. The output field of view image is obtained. The output field of view The specific calculation formula is as follows: ; in, This represents the input image field of view, i.e., the image in the virtual environment; 'a' represents the digital magnification. This indicates that the field of view is cropped proportionally. This indicates resampling to the target resolution.

[0027] In step A2, the output field of view image obtained in step A1 is... First, dynamic range mapping is performed, preferably Gamma correction mapping, to obtain the Gamma-corrected image. ; In step A3, the Gamma-corrected image obtained in step A2 Based on this, perform CLAHE local contrast enhancement to obtain the photometric mapping result image. ; Image after Gamma correction mapping Defined as: ; in Indicates in The intermediate result image obtained after Gamma correction mapping at each time step; (·) denotes the Gamma correction operator with κ as the exponent; Pixel value; This indicates truncation to the valid range [0, 1]. This embodiment uses an 8-bit image as input; κ is the Gamma correction coefficient (or gamma value). It is preferred for enhancing details in dark areas. It is preferred for darkening dark areas to enhance the detail of bright areas.

[0028] This CLAHE local contrast enhancement is denoted as Its reinforcing strength is determined by parameters Control. Preferably, it reduces color shift; CLAHE acts on the luminance channel (e.g., LAB). Channel or YUV After local contrast enhancement, the chroma channel is combined with the chroma channel to return to RGB, resulting in a photometric mapping image. Among them, the photometric mapping result image The output can be represented as: ; in, The overlay intensity is used to control the local contrast enhancement, thereby controlling the brightness difference and recognizability between high-frequency textures and the background.

[0029] In step A4, the photometric mapping result image from step A3... By superimposing high-frequency edge information on the original image, an edge-enhanced image is obtained. Preferably, the edge enhancement is implemented using the Laplacian operator or its equivalent high-pass enhancement.

[0030] Preferably, as an equivalent implementation of the Laplacian family of edges, an anti-masking sharpening method using Gaussian smoothing difference can be employed to obtain an edge-enhanced image. Among them, edge enhancement image Specifically, it is expressed as follows: ; in, This represents the Gaussian filter operator. The standard deviation of the Gaussian filter is used to control the bandwidth of the smoothing kernel. The larger the value, the wider the ambiguity range; Indicates edge enhancement intensity.

[0031] Alternatively, the edge enhancement image can also be represented using a discrete Laplacian convolution. Specifically: ; in, For the Laplace operator.

[0032] In step A5, the edge enhancement image obtained in step A4 is used. Temporal resampling is performed to simulate the physical limitations of visual aids, thereby obtaining an auxiliary preprocessed image. The image to be processed, wherein the temporal resampling processing logic specifically includes: (1) Delayed modeling: The output image comes from The processing result at any given moment. Delay. It is measured in milliseconds and is implemented by caching historical frames through a ring buffer, that is, reading frame data that is lagging behind the current time from the buffer at the time of display.

[0033] (2) Refresh rate limit: Limit the output update to the refresh rate. (Hz), at refresh time Sampling and frame-holding display are used to create a stepped update effect, aiding in image preprocessing. It can be represented as: ; in This is to refresh the sampling time. When the input processing frame rate is higher than the refresh rate, the frame rate is reduced by repeating the previous frame or skipping the intermediate frame, thereby simulating the stuttering and discontinuity of motion when dropping from 90Hz to 30Hz.

[0034] Preferably, the step of performing pathological visual perception processing on the image to be processed to obtain the target output image specifically includes: B1. Based on the sensitivity mask and the frequency band loss coefficient, perform multi-channel Mipmap sampling on the image to be processed to construct low-frequency channel, mid-frequency channel and high-frequency channel, and reconstruct the spatial frequency degraded visual image according to preset or dynamically determined weights; B2. Calculate the retinal slip angular velocity and time frequency based on key information of scene objects, generate pathological weights by combining the retinal slip angular velocity and time frequency with the time contrast sensitivity model parameters, and use the pathological weights to modulate time integration, contrast compression and significance attenuation. B3. For the visually defective region, perform low-frequency texture sampling based on the defect boundary and smooth sampling coordinates to obtain the target visual fill color, and write the target visual fill color into the frame buffer to obtain the target output image.

[0035] In step B1, a real-time visual simulation technique based on frequency channel decomposition is proposed. This technique utilizes the mipmap capabilities of a graphics processing unit (GPU) to logically decompose a single rendered image in a virtual environment into three independent frequency bands: low-frequency, mid-frequency, and high-frequency, through a multi-channel mipmap blending algorithm. The level of detail (LOD) of each frequency band is adjusted according to the patient's pathological parameters, selectively attenuating different spatial frequency information according to sensitivity masks and frequency band loss coefficients. This simulates the pathological changes in the contrast sensitivity function (CSF) of the human eye and yields the target visual image. The multi-channel mipmap blending algorithm simulates the loss of spatial detail caused by P-path damage through multi-channel frequency decoupling technology, achieving hierarchical visual degradation consistent with the physiological mechanisms of the human eye. While suppressing details such as high-frequency stripes and dense checkerboard boundaries, the algorithm preserves larger-scale geometric contours and low-frequency brightness structures, restoring the realistic perceptual state of "blurred vision, yet clear form" experienced by patients with central visual field impairment. Figure 4 As shown, where, Figure 4 (a) is Figure 3 The input image after device preprocessing contains geometric shapes of different sizes, checkerboard patterns, and stripes of different spatial frequencies; Figure 4(b) is the central visual sensitivity mask, which is used to define the visual sensitivity of each pixel in the screen space. In the figure, the central sensitivity approaches 0 and gradually transitions to 1.0 towards the periphery. Figure 4 (c) Showing the output of the traditional single-channel overall blur algorithm, in the low-sensitivity central region, high, medium and low frequency information are blurred together without difference, and the local structure degenerates into uniform color blocks; Figure 4 (d) The output results of the multi-band selective degradation algorithm of this invention are shown. Under the same sensitivity mask, high-frequency details are preferentially suppressed, while larger-scale geometric contours, low-frequency brightness variations, and some mid-frequency structures are still preserved. (Comprehensive comparison) Figure 4 (c) and Figure 4 (d) It can be seen that the multi-channel Mipmap hybrid algorithm of the present invention can more accurately express the non-uniform degradation characteristics of different spatial frequency channels in pathological vision.

[0036] In step B2, retinal slippage velocity is calculated using a dynamic contrast algorithm based on the characteristics of the peripheral visual field (M-pathway), and the dynamic contrast is modulated in real time accordingly. This step can simulate motion perception impairment, decreased dynamic contrast, and loss of target salience caused by optic nerve damage, thereby improving the clinical effectiveness of the system in assessing dynamic interactive tasks. Figure 5 As shown in the figure, this diagram combines pathological weighting maps, quantitative curves, and visual rendering results to demonstrate the comparative effect of temporal contrast modulation processing. Figure 5 (a) is a pathological weighting map of the peripheral field of vision, used to represent the spatial relationship between the eccentric angle, local edge salience and temporal modulation intensity; Figure 5 (b) Display the RMS contrast ratio curves of low-frequency, mid-frequency and high-frequency components when the target moves laterally in the field of view. The curves show a gating feature with weaker suppression in the central region and stronger suppression in the peripheral region, and reflect the difference in suppression intensity in different frequency bands. Figure 5 (c) and Figure 5 (d) The output image with spatial frequency degradation and the output image after further temporal modulation are shown respectively when the target is in the peripheral field of view. Motion blur, contrast compression and saliency attenuation of the peripheral target can be observed. Figure 5 (e) and Figure 5 (f) shows the spatial frequency degradation output image and the temporal modulation output image when the target is close to the central field of vision, respectively. At this time, the pathological gating weight is small, and the structure of the central region remains relatively stable. It can be seen that the present invention can reflect the differentiated temporal pathological features of decreased peripheral motion perception and relatively preserved central field of vision in dynamic EVA example sequences.

[0037] In step B3, a real-time image inpainting algorithm is used to recreate the "negative scotoma" phenomenon observed in patients with glaucoma or macular degeneration. This phenomenon is not only a psychophysical phenomenon but also an inference strategy formed by the cerebral cortex (V1 / V2 areas) when processing visual information. The real-time image inpainting algorithm differs from traditional simple black spot occlusion methods by employing a combination of boundary anchoring, low-frequency texture sampling, and local structural extension to simulate the cerebral cortex's perceptual filling mechanism for visual field blind spots. This reduces the distortion caused by traditional simulation methods in assessing patient recognition ability against complex backgrounds. Figure 6 As shown in the figure, based on an EVA example image, this diagram demonstrates the comparative performance of the algorithm of this invention with traditional methods under three pathological features: central scotoma in macular degeneration, arcuate scotoma in glaucoma, and nasal stepladder in glaucoma. (First row) Figure 6 (a) to Figure 6 (e) Corresponding to central scotoma in macular degeneration, second row Figure 6 (f) to Figure 6 (j) corresponds to the arcuate scotoma in glaucoma, third row. Figure 6 (k) to Figure 6 (o) Corresponds to the nasal step in glaucoma. In each row, the first column is the clinical visual field sensitivity distribution, the second column is the overlay reference of the EVA example image and the defect area, the third column is the result of traditional black spot occlusion, the fourth column is the result of low-pass blur filling baseline, and the fifth column is the result of the perceptual filling of this invention. Comparing the third, fourth and fifth columns, it can be seen that traditional black spot occlusion introduces a visible black boundary cue, and low-pass blur filling can only retain large-scale brightness and color trends while easily losing lines, arcs and stripe structures; the perceptual filling of this invention, through structural extension and low-frequency texture sampling near the defect boundary, makes the defect area visually present a "no visible black spot, structural continuity but uncertain details" effect, which is more in line with the visual experience of patients under negative scotomas who are not easily aware of the defect area.

[0038] In this embodiment, pathological visual perception processing includes spatial frequency degradation processing, temporal contrast modulation processing, and visual defect area perception filling processing, corresponding to steps B1, B2, and B3, respectively. Based on the information processing mechanisms of the P and M pathways in visual neuroscience, and combined with clinical visual field sensitivity distribution, contrast sensitivity, and temporal contrast sensitivity data, a pathological visual perception model for VR environments is constructed. This model simulates selective loss of spatial details through spatial frequency degradation processing, simulates motion perception sluggishness, dynamic contrast decrease, and significant attenuation through temporal contrast modulation processing, and simulates the "seeing but not seeing" phenomenon of defect areas through visual defect area perception filling processing, thereby more realistically reproducing pathological visual features while ensuring real-time performance.

[0039] Preferably, the step of performing multi-channel Mipmap sampling on the image to be processed based on the sensitivity mask and the frequency band loss coefficient, constructing low-frequency, mid-frequency, and high-frequency channels, and reconstructing the spatially frequency-degraded visual image according to preset or dynamically determined weights, specifically includes: C1. Sample the sensitivity mask based on the texture coordinates of the current pixel to obtain the local visual sensitivity; C2. Calculate the target blurring degree of the low-frequency channel, mid-frequency channel, and high-frequency channel respectively based on the local visual sensitivity and the frequency band loss coefficient; C3. Perform three parallel samplings on the scene texture according to the target blur level to obtain the color components of the low-frequency channel, the mid-frequency channel and the high-frequency channel; C4. Linearly superimpose the color components and preset or dynamically determined weights to obtain the visual image.

[0040] In step C1, the sensitivity mask is sampled based on the scene texture coordinates of the current pixel to obtain the local visual sensitivity. Here, the scene texture (_MainTex) represents the input image for the current pathological visual processing stage, i.e., the image to be processed obtained in step A5 when the visual aid device model is enabled; the sensitivity mask (_Overlay) is a grayscale texture used to define the visual sensitivity of each pixel in screen space, with values ​​ranging from [value range missing]. Where 1 represents normal sensitivity and 0 represents a complete blind zone; this sensitivity mask is derived from the pathological parameters of the object to be evaluated. This sensitivity mask is preferably generated from clinical visual field examination results, lesion region annotation results, or a sensitivity prior template constructed based on disease type and severity, and then registered in screen space to form a grayscale texture _Overlay consistent with the resolution of the current rendering frame; Furthermore, in the fragment shader, based on the scene texture coordinates of the current pixel... The local visual sensitivity of a pixel is obtained by sampling the sensitivity mask (_Overlay). , ,Right now:

[0041] in, Represents a two-dimensional texture sampling function; This indicates the first channel of the color vector (usually the four channels of RGBA) returned by the sample. Since the "sensitivity mask (_Overlay)" is a grayscale texture, it actually extracts the grayscale value of that point.

[0042] In step C2, based on local visual sensitivity The target blur level (LOD) of the low-frequency, mid-frequency, and high-frequency channels is calculated using the band loss coefficients (_Loss params) to form low-pass sampling layers with three different cutoff scales on the Mipmap pyramid, thereby constructing low / mid / high-frequency channels. Here, _HighFreqLoss is the high-frequency loss coefficient, reflecting the attenuation of high-frequency details; _MidFreqLoss is the mid-frequency loss coefficient, reflecting the attenuation of mid-frequency contours; and _LowFreqLoss is the low-frequency loss coefficient, reflecting the attenuation of low-frequency structures. All three values ​​are in the range [0,1] and are determined by the contrast sensitivity-related pathological parameters of the object being evaluated.

[0043] The degree of ambiguity of the target is calculated in the following form (exemplary segmentation ranges are 0–2, 2–5, and 5–7; specific segmentation ranges can be determined according to actual circumstances): High frequency layer ( ): Corresponds to Mipmap 0–2 layers, used to maintain a higher resolution (as an upper bound low-pass for the high-frequency residual signal):

[0044] Intermediate frequency layer ( ): Corresponding to Mipmap 2–5 layers:

[0045] Low-frequency layer ( ): Corresponds to Mipmap layers 5–7 (stronger low-pass filter, used to preserve low-frequency structures such as outlines / color blocks):

[0046] in The mathematical expression can be represented as:

[0047] Furthermore, to ensure the stability and monotonicity of the frequency band construction, it is preferable to incorporate LOD order constraints:

[0048] For example, it can be done through:

[0049] in To prevent small positive numbers (e.g., 0.01) from causing residual channel degradation due to equal values.

[0050] It is important to note the local visual sensitivity. When the value is smaller (the more severe the lesion) or the loss coefficient is larger, the overall LOD value tends to be at a higher level (more ambiguous), thereby achieving selective attenuation of local spatial frequency information.

[0051] In step C3, the GPU's tex2Dlod (or SampleLevel) instruction is used to perform three parallel samplings on the same scene texture _MainTex based on the three LOD values ​​mentioned above, obtaining the low-pass sampling results for the low-frequency channel, mid-frequency channel, and high-frequency channel, respectively denoted as... : ; ; ; Based on this, differential reconstruction (using the Laplacian pyramid concept) is preferred to explicitly decompose the image into three frequency bands: low frequency, mid frequency, and high frequency, thus explicitly constructing the frequency band channels using differential reconstruction. Low-frequency channel color components : ; Intermediate frequency channel color components : ; High-frequency channel color components : ; Using the above method, three frequency bands of color components can be obtained using only three texture samples, where... It primarily represents high-frequency information such as text details, fine lines, and sharp edges; The main features are outline and mesoscale texture; It mainly characterizes the low-frequency components of large structure and color / brightness.

[0052] In step C4, the color components of the low-frequency, mid-frequency, and high-frequency channels are linearly superimposed according to preset or dynamically determined weights (or frequency band gains) to obtain a visual image. : ; The dynamically determined weight is preferably calculated based on the local sensitivity and the frequency band loss coefficient, and is used to reflect the degree of differential preservation of low-frequency, mid-frequency and high-frequency information in different lesion areas. All represent preset weights, preferably based on local sensitivity. The frequency band loss coefficients (_Loss params) are dynamically determined to simulate the attenuation of pathological CSF (contrast sensitivity function) in different frequency bands. For example, the following values ​​can be taken: ; ; ; This allows for the following to be achieved: in the central lesion area ( It significantly suppresses high-frequency and even mid-frequency details, while relatively preserving low-frequency structure and color information, thus creating a realistic perceptual effect of "seeing unclearly but having form even when blurred".

[0053] Furthermore, the above differential reconstruction possesses verifiable reconstruction consistency: when At that time, visual images It can be represented as: ; And local sensitivity hour The corresponding layer is approximately the original layer (e.g., LOD=0), so the output is consistent with the original image, which helps to avoid introducing unnecessary image quality deviations in non-lesion areas.

[0054] Traditional frequency simulation methods typically rely on FFT or large convolutional kernels, which are computationally intensive and bandwidth-intensive, making them difficult to run stably in real-time rendering scenarios such as mobile VR. This embodiment utilizes the built-in Mipmap chain and LOD sampling capabilities of the GPU hardware to complete multi-scale low-pass acquisition at the fragment level with only 3 texture samples. By constructing low / medium / high frequency channels through a small amount of differential and weighted operations, it achieves selective attenuation of different spatial frequency information.

[0055] It should be noted that the computational overhead of this scheme is a fixed constant at the per-pixel level (constant sampling number and stable instruction path), and the overall frame overhead increases linearly with the number of pixels, making it more suitable for mobile devices and immersive interactive applications with strict real-time requirements.

[0056] Preferably, the step of calculating the retinal slip angular velocity and temporal frequency based on key information of scene objects, generating pathological weights by combining the retinal slip angular velocity and temporal frequency with the parameters of the temporal contrast sensitivity model, and using the pathological weights to modulate temporal integration, contrast compression, and significance attenuation, specifically includes: D1. Input the world coordinate velocity, distance and geometric boundary information of scene objects, the user's gaze direction, observer, camera pose information and preset parameter data; D2. Map the world coordinate velocity in combination with the observer or camera pose information onto the retinal coordinate system to obtain the projected angular velocity; D3. Calculate the time frequency based on the projected angular velocity and the target's local spatial frequency characteristics; D4. Calculate the time sensitivity ratio based on the individualized damage coefficient and the time-comparison sensitivity model; D5. Perform nonlinear gating mapping on the time sensitivity ratio to generate normalized pathological weights; D6. Modulate time integration, contrast compression, and significance attenuation according to the pathological weights; The key information includes: world coordinate velocity, distance and geometric boundary information, user's gaze direction, observer, camera pose information, and preset parameter data.

[0057] Step D1 involves multi-source data input. The following data is obtained as algorithm input: (1) World coordinate velocity of scene objects ,distance and geometric boundary information ; (2) User's gaze direction (Preferably a unit vector) ); (3) Clinical parameter data related to the visual function of the subjects, including flicker or frequency doubling test results (time contrast sensitivity related indicators). (4) Observer or camera pose information: eye or camera position Object position and observer / camera speed ; (5) Preset parameter data, including but not limited to: Time-comparison sensitivity model parameters (normal reference model parameters, patient model parameter fitting coefficients, etc.); Nonlinear gated parameters (e.g.) ); Peripheral view gating parameters (e.g.) ); Rendering modulation parameters (e.g.) wait).

[0058] Furthermore, the world coordinate velocity, distance, and geometric boundary information of scene objects in step D1 are preferably output in real time by the virtual environment engine. The world coordinate velocity can be obtained from the object's rigid body velocity, displacement increment, or animation-driven velocity; the distance and geometric boundary information can be obtained from the object's bounds, depth buffer, or collider bounding box. The user's gaze direction and the observer / camera pose are preferably acquired in real time by the head-mounted display pose tracking module and eye-tracking module. Clinical parameter data related to the subject's visual function are preferably derived from flicker sensitivity tests, frequency doubling detection, time contrast sensitivity tests, or preset model parameters obtained by mapping disease type and severity. The fitting coefficients of the normal reference model parameters and the patient model parameters are used in step D4 to construct the time contrast sensitivity response and output the time sensitivity ratio. Nonlinear gating parameters and peripheral vision gating parameters are used in step D5 to map the time sensitivity ratio to normalized pathological weights. Rendering modulation parameters are used in step D6 to control the motion blur length, contrast compression intensity, and significant attenuation intensity. The visual image output in step C4 is used as the input image for steps D2 to D6. In step D2, the world coordinate velocity, observer pose, and gaze direction jointly determine the retinal slip angular velocity. In step D3, the retinal slip angular velocity and the local spatial frequency estimated from the target boundary size and texture detail density are used to calculate the temporal frequency. In step D4, the temporal frequency is used as an index to query the sensitivity response of the normal model and the patient model, and the temporal sensitivity ratio is output. In step D5, the temporal sensitivity ratio is mapped to normalized pathological weights through nonlinear gating and eccentric angle gating. In step D6, the pathological weights are used as control variables for three rendering branches: motion blur, dynamic fading, and saliency attenuation, respectively. Temporal domain modulation is performed on the pixels of the target region, and the target pathological visual image is output.

[0059] Step D2 involves mapping the world coordinate velocity to retinal coordinates and calculating the angular velocity. Based on the principles of geometric optics, the object's world coordinate velocity, combined with the observer's or camera's pose information, is mapped to the retinal coordinate system to calculate the projected angular velocity. (Unit: deg / s). This projected angular velocity is used to characterize the slip intensity of a moving target on the retina, reflecting the true dynamic stimulus intensity to the retinal photoreceptors / receptive fields. The calculation process is as follows: (1) First define the unit vector of the line of sight. : ; (2) Using relative velocity : ; (3) Extract the velocity component perpendicular to the line of sight. : ; (4) Then the projection angular velocity (retinal slip angular velocity) The preferred calculation is as follows: ; in, Used to convert rad / s to deg / s.

[0060] In a similar form, it can also be written as: ; in, Let the angle between the direction of motion and the direction of the line of sight satisfy... .

[0061] The above form is equivalent to decomposing "world speed" into components along the line of sight and perpendicular to the line of sight, and using the perpendicular component to determine retinal slip; this definition effectively avoids relying solely on... Introduced directional error.

[0062] Step D3 is the time frequency calculation, which involves calculating the angular velocity. By combining the target's local spatial frequency characteristics, the corresponding temporal frequency is calculated. This time frequency The rate of change of light intensity signal within a specific receptive field of the retina describes the temporal rate of change and is a key indicator determining the response of the M-pathway. The time-frequency response satisfies: ; in, The local spatial frequency of the target from the observation perspective (unit: cycles / deg). Fast estimation based on Bounds projection: Based on the target geometric boundary With distance Estimate target angular size ,For example: ; in, The physical dimensions of the target in the world coordinate system (units consistent with the scene coordinate system) can be obtained from the principal direction span of the target's geometric boundaries Bounds; To prevent division by zero of tiny positive numbers.

[0063] And introduce texture density parameters , This represents the equivalent texture cycles of the target along its main viewing direction, measured in cycles / object-width, and can be preset or looked up from a table based on the material type; local spatial frequency. Convert to cycles / deg using the following formula: ; Step D4 is the time sensitivity ratio. The calculation generates individualized injury coefficients based on input clinicopathological parameters and, combined with a pre-established time-contrast sensitivity model, calculates the patient's injury rate over time. The sensitivity value is calculated below and then compared with the normal reference model to obtain the time sensitivity ratio. Among them, the time sensitivity ratio The specific calculation formula is as follows: ; in, For patients in terms of time and frequency The time-comparison sensitivity response value is obtained by fitting the individualized damage coefficient with the parameters of the time-comparison sensitivity model. For the normal reference population at a certain time frequency The time-comparison sensitivity response value is determined by the parameters of the normal reference model. Step D5 is the time sensitivity ratio. Perform nonlinear gating mapping (e.g., using the Sigmoid function) to generate normalized pathological weights. In this process, when performing nonlinear gating mapping on the time sensitivity ratio, a peripheral field-of-view gating weight based on the eccentric angle is introduced. Furthermore, the directional blur length, contrast compression intensity, and significance attenuation intensity are modulated according to the pathological weight. The directional blur length is a saturation function that increases with increasing projection angular velocity, and the modulation only applies to the target region or the target projection region. The gating satisfies a monotonic relationship: "the lower the sensitivity, the stronger the pathological effect." The smaller, The larger the value, the greater the pathological weight can be expressed as: ; in This is the abnormal threshold. This is the transition slope parameter.

[0064] More preferably, to highlight the algorithm's role in the peripheral field of view (the M-path dominant region), an eccentric angle-gated weight can be introduced. eccentric angle It can be calculated from the direction of gaze and the direction of eye contact: ; And set up perimeter gating: ; This is a smooth step transition function. When the input value is less than the first parameter, it outputs 0, and when it is greater than the second parameter, it outputs 1. Between the two, it smoothly transitions from 0 to 1 according to a cubic polynomial. The final pathological weights are preferably: ; Introduction This allows the central visual field to maintain minimal distortion, while the peripheral visual field experiences a more significant decrease in dynamic contrast and reduced motion perception, thus aligning with the algorithm's localization.

[0065] Then, using weights Simultaneously modulate the following perceptual effects: (1) Temporal integration modulation (motion blur): Simulates the extension of the integration time in a visual system. Based on weights... Increasing the step size / length of directional blur sampling produces a motion blur effect for moving objects. Preferably, the blur length... It can be satisfied: ; in, Based on the fuzzy length, For the maximum additional blur length, This is the reference angular velocity. A saturation function, used to restrict the input value to a certain range. Within the specified interval, when the target angular velocity exceeds the reference value, the blur length no longer increases indefinitely, thus ensuring rendering stability and visual controllability. Its mathematical definition is: ; (2) Contrast Compression Modulation (Dynamic Fading): Simulates dynamic contrast reduction. Based on weights. This involves linearly interpolating the target pixel's color or brightness to the ambient brightness baseline (lerp), reducing its brightness contrast or color saturation, as shown below: ; in The background baseline (preferably the local neighborhood mean or low-pass filter result) is used. For the target pixel color or brightness value, The intensity of fading.

[0066] (3) Significance attenuation modulation (motion silence): Simulates the "stealth" phenomenon of a target in the attention mechanism. Based on weights Reducing the saliency of the target rendering, making it less noticeable in complex dynamic backgrounds, is represented as: ; in This is the significant attenuation coefficient; alternatively, the significant channel, stroke, or highlight intensity can also be set according to... attenuation, To perform significant attenuation on the target pixel color (or brightness).

[0067] In step D6, at the end of the rendering pipeline (fragment shader or post-processing stage), the modulation effect described above is fused with the current image entering step D6 to output the final pathological visual image. To avoid unnecessary distortion in non-target areas, modulation only applies to the target area (or target projection area) and is achieved through a target mask. Complete pixel-level fusion with the original image, target mask. It is generated by combining the geometric boundary information of scene objects with depth / template buffers or object instance identifiers.

[0068] 1) Motion blurring yields the first intermediate image. For any pixel position p in the image, the unit vector along the direction of motion... sampling Next, we obtain: ; in, Let be a pixel offset vector, satisfying Step length The pixel offset step size is determined by the total blur length. L and number of samples N Decision, satisfaction: ; If we consider the target mask (to avoid sampling the background), we can perform mask weighting and normalization on the samples: ; in This represents the specific pixel coordinates of the i-th sample; Indicates the target mask at pixel position value at To prevent extremely small values ​​from being divided by zero, such as 10 -6 .

[0069] 2) Contrast compression yields the second intermediate image. In obtaining the first intermediate image Then, dynamic contrast compression (dynamic fading) is performed on the target area. First, a background brightness baseline / ambient baseline image is defined. (Can be obtained by low-pass filtering or neighborhood averaging of the original image). For any pixel location within the target region. p The corresponding second intermediate image Obtained by weighted interpolation using the following formula: ; in, Represents a linear interpolation function; For the contrast compression interpolation weights at the corresponding pixel positions, the following condition must be met: ; The contrast compression strength coefficient. This represents a truncation function used to restrict the calculation result to the interval [0,1].

[0070] It should be noted that if implemented in the grayscale / luminance domain, it can be Replace with luminance component ,right Perform the above interpolation, and then write back to the color space.

[0071] 3) Final pathological target image obtained by significant attenuation

[0072] Second intermediate image Further, significant attenuation is performed on any pixel location within the target region. p The corresponding final pathological target image It is preferable to use the direct expression of "energy / amplitude reduction": ; in, The significance attenuation weight for the corresponding pixel position satisfies: ; in, For pixel position p Normalized pathological weights at the location; The significance attenuation coefficient; This represents a truncation function used to restrict the calculation result to the interval [0, 1].

[0073] 4) Calculation : As the final fusion output of step D6, the final pathological visual image at any pixel location The value at the location It is through the target mask Pathological target results With the original scene Pixel-level fusion yielded: ; in, This refers to a fragment corresponding to a screen pixel position or texture coordinate. To render the original scene in the pipeline at positionp The color (or brightness) of the area; The color (or brightness) of the pathological target after applying "motion blur, contrast compression, and saliency attenuation" to the target area; For the target area mask at position p The weight value at the location satisfies :when When this occurs, it indicates that the pixel completely belongs to the target area, and the output fully adopts [the specified method / applicability]. ;when When this condition is met, it indicates that the pixel belongs entirely to the non-target area, and the output retains all of these values. ;when This is often seen in anti-aliased edges, semi-transparent or soft boundary situations, to achieve a smooth interpolation transition between the two, avoiding target edge breakage and flickering.

[0074] To avoid introducing unnatural brightness abrupt changes when the target edge blends with the background, It can generate soft masks using target coverage, depth / template buffering, or edge-based methods, and can perform temporal smoothing filtering to improve stability.

[0075] In a verification simulation experiment, when the target moves at a given screen angular velocity and the eccentric angle gradually increases, the temporal contrast modulation processing of this invention exhibits differentiated suppression of different spatial frequency bands in the peripheral region. The suppression of mid-frequency and high-frequency components is generally stronger than that of low-frequency components, resulting in a significant decrease in the visibility of the dynamic target in the peripheral field of view, weakened edge details, and relatively preserved low-frequency contours. This phenomenon is consistent with the design goal of this invention, which modulates dynamic contrast using a time-frequency sensitivity model, and can be used to support the assessment of pathological motion perception defects in dynamic interactive tasks such as driving and obstacle avoidance. This step ensures that users experience a visual effect consistent with the real pathological state of "not seeing" or "blurred and indistinguishable" when observing dynamic objects, and improves the clinical effectiveness of the system in the assessment of dynamic interactive tasks such as driving and obstacle avoidance.

[0076] Preferably, the step of performing low-frequency texture sampling based on the defect boundary and smoothed sampling coordinates for the visually defective region to obtain the target visual fill color, and writing the target visual fill color into the frame buffer to obtain the target output image, specifically includes: E1. Define the visual defect region and calculate the defect boundary based on the defect mask; E2. Determine the boundary anchor points based on the defect boundary, and calculate the smoothed sampling coordinates based on the nonlinear weighting function; E3. Perform low-frequency texture sampling based on the smoothed sampling coordinates to obtain the target visual fill color of the defective area; E4. Write the target visual fill color into the frame buffer to obtain the target output image.

[0077] In step E1, the following data needs to be entered: 1. Input texture : Represents the texture of the pathological visual image output in step D6; : Represents the texture (single channel) of the visual field defect mask, where 0 represents the normal visual field area; 1 represents the visual defect area, which is generated by mapping the clinical visual field sensitivity distribution from screen pixels to visual field angle coordinates and thresholding, and corrected for the temporal and nasal directions according to the relationship between the left and right eyeglass images.

[0078] 2. Configuration parameters: Maximum Mipmap level, using This indicates the upper limit of the ambiguity level used to control low-frequency sampling; The shape parameter of the nonlinear weighting function controls the strength of the "edge anchoring". When enhancing edge outward thrust is not required, it can be set to... =1.

[0079] The central degradation intensity coefficient is used to enhance the pathological realism of lost central information. When it is not necessary to simulate severe loss of central information, it can be set to a lower value. =0.

[0080] Sensitivity distributions can be generated from clinical visual field examination styles to improve the clinical consistency of pathology simulations. Screen pixels are mapped to visual field angular coordinates, and a sensitivity distribution is constructed within these coordinates. Then, the defect area is obtained through thresholding: 1. Pixel → Angle Mapping: Let the horizontal field of view be The image width is Image height is H Then the angle per pixel is Corresponding pixels The angular coordinates can be: ; 2. Left and right eyeglass images (OD / OS): To reflect the mirror image relationship between the temporal and nasal sides in the clinical visual field of the left and right eyes, it is preferable to define standardized coordinates so that "temporal side" is the positive direction. Therefore, for the left eye's OS, the following can be used: ; For the right eye OD, the following method is used: ; 3. Sensitivity thresholding generates defect areas: Let the threshold be (For example Then we can define: ; From this, we obtain It can present defect morphologies closer to the Humphrey VF style (such as a 24-2 dot matrix distribution) and naturally include typical structures such as blind spots, arched dark spots, nasal step-like structures, and central dark spots. Then, it determines pixel-by-pixel whether the currently processed location is within the visual defect area. For any screen pixel... : like Output the original image colors. ; like Enter the inference path and execute steps E2–E4.

[0081] Furthermore, the field defect mask ScotomaMask in step E1 is preferably derived from the sensitivity distribution of clinical visual field examination, the results of fundus lesion segmentation, or a defect template generated from parameters such as lesion center, morphology, and severity; the coordinates of the defect area center are... and equivalent radius It can be obtained from the centroid and area of ​​ScotomaMask, or directly given by clinical annotations; maximum Mipmap level The preferred method is to determine the shape parameters of the nonlinear weighting function based on the display resolution, the defect scale, and the lowest spatial frequency to be retained; and the central degradation intensity coefficient The preferred method is determined based on the type of disease, the depth of the defect, or the results of empirical calibration.

[0082] Preferably, in step E1, the ColorTexture is taken from the pathological visual image output in step D6, so that perceptual filling directly applies to the result after spatial frequency degradation and temporal modulation have been completed. Specifically, when the visual aid device model is not enabled, the pathological visual image originates from the output obtained by performing steps C1 to C4 and steps D1 to D6 on the original rendering frame of the virtual environment; when the visual aid device model is enabled, the pathological visual image originates from the output obtained by performing steps C1 to C4 and steps D1 to D6 on the image to be processed obtained in step A5. Subsequently, during pixel-by-pixel processing, in step E1, it is first determined whether the current pixel enters the defect filling path based on ScotomaMask; if it does, in step E2, ScotomaMask is input into the distance transformation and gradient calculation process to obtain the boundary anchor point, normalized depth, and smoothed sampling coordinates. In step E3, and Together, they serve as control inputs for low-frequency texture sampling, extracting the base value of the target visual fill color from the ColorTexture; in step E4, further based on... The normalized depth is used to perform center degradation enhancement on the target visual fill color, and the resulting fill result is written to the frame buffer to form the final pathological visual output image for evaluation.

[0083] In step E2, the defective region is not strictly circular (e.g., the arcuate scotoma of glaucoma, the nasal step, etc.). To make the algorithm applicable to scotomas of arbitrary shapes... Instead of relying on a single geometric model, it calculates the distance field from the inside of the defect to the boundary based on the defect mask and constructs the boundary anchor points accordingly. Preferably, the distance field is defined within the defect region. For pixels The distance to the nearest normal region boundary (which can be obtained through distance transformation). Then: 1. Direction of the outer normal to the boundary: ; in Represents the Euclidean norm; This represents the gradient operator.

[0084] 2. Boundary anchor points: ; 3. Normalized depth and substitution radial quantity: set up Then we can define: ; At the boundary Deepest part of the center .

[0085] 4. Smooth sampling coordinates : ; in, .

[0086] Preferably, Let be a monotonically increasing nonlinear weighting function, such that when (Near the center) ,when (Near the boundary) More preferably, A smooth function with first-order continuity can be used, for example: ; in This can enhance the extroverted nature of "edge anchoring". Indicates truncation to .

[0087] This step can maintain the consistency of the perceptual filling mechanism of "boundary anchoring → pushing inward and outward → low-frequency uncertainty" even under irregular defect morphology, and avoid the recognition and evaluation distortion caused by simple black spot occlusion in complex background.

[0088] In step E3, low-frequency texture sampling is used to simulate the brain's probabilistic guesses about missing information as "fuzzy but plausible." Example pseudocode is as follows: float3 color_low = SAMPLE_TEXTURE2D_LOD( ColorTexture, sampler_LinearClamp, , LOD = ); The above process is performed at a specified level of detail (LOD= A 2D texture is sampled, high-frequency details are automatically filtered out, and only the average tone and low-frequency structure of the area are retained, thereby expressing the uncertainty and "reasonable fuzziness" of the missing area information.

[0090] In step E4, the obtained low-frequency sampled color is used as the basis for the target visual fill color and written to the frame buffer. Degradation enhancement can be applied to the central region of the defect to improve the realism of the pathological simulation, for example, by reducing the brightness or contrast of the central region. The calculation method is as follows: ; in, The attenuation intensity coefficient, Characterizing the degeneracy weights from the boundary to the center: when The decay approaches 0 at the boundary. The attenuation reaches its maximum at the center, which is used to simulate the hollow feeling caused by severe loss of information in the fovea and the transition effect between the defective edge and the normal field of vision.

[0091] Finally, the calculated The output color of that pixel is written into the frame buffer, thus obtaining the target output image from which the evaluation results are obtained.

[0092] The evaluation results are obtained by performing a standardized virtual task on the target output image; the target output image is tested with a standardized virtual task, and at least one of the following is collected: task completion time, number of collisions, gait parameters, head movement parameters, hand movement parameters, eye movement gaze heatmap, gaze duration and saccade time, and the evaluation results of the visual aid device's assistive effect are output.

[0093] In practical applications, standardized tasks, such as shopping item finding and road obstacle avoidance, are loaded into the "Visual Behavior Simulation and Analysis Management System," and behavioral results are recorded under both states. The preferred gait parameters include at least one of gait speed, stride length, stride length standard deviation, stride length coefficient of variation, and obstacle-crossing foot lift margin; the preferred head movement parameters include at least one of head pitch variation amplitude, horizontal scan amplitude, and angular velocity; and the preferred eye movement parameters include eye fixation heatmap, fixation duration, and saccade duration.

[0094] In practical applications, the task execution result of the target output image in a standardized VR task is as follows: Figure 7 As shown. Figure 7 The study presented a multidimensional objective quantitative assessment report generated after subjects completed a standardized virtual task. This report was used to evaluate visual search strategies, motor control characteristics, and the assistive effects of visual aids in pathological visual states. Figure 7 (a) Shows the feedback of eye movement parameters in the supermarket item search scenario, including eye movement gaze heatmap, gaze duration and saccade time. Figure 7 (b) Shows task performance feedback in outdoor walking scenarios, including result indicators such as task completion time and number of collisions. Figure 7 (c) shows the vertical height-time curves of the left and right heel markers within the Z-axis cutoff region, with solid dots marking heel bottoming events; Figure 7 (d) shows the vertical height-time curves of the left and right toe markers, with solid triangles marking toe-off events; by combining... Figure 7 (c) and Figure 7 (d) The complete gait cycle can be divided. Figure 7 (e) is a bar chart of the step length distribution on one side during a single straight-line process, with the red line representing the average step length of the walk; Figure 7 (f) is a summary panel of core quantitative indicators of gait, which displays indicators such as average stride length, standard deviation of stride length, stride speed and coefficient of variation of stride length.

[0095] As a set of comparative implementation methods, the scheme of the present invention can be compared with the simple occlusion method and the simple blurring method under the same VR task and the same pathological parameters. The simple occlusion method directly displays the defect area as a visible black spot or occlusion block, which easily introduces the bias of complete occlusion and explicit boundary indication at the same time; the simple blurring method is difficult to reflect the selective loss of local spatial frequency, the decrease in surrounding dynamic contrast, and the perceptual filling effect at the same time. Based on the cascade processing mechanism of spatial frequency degradation, temporal modulation and perceptual filling adopted by the present invention, in tasks such as obstacle avoidance, object finding and street crossing, the scheme of the present invention is generally more likely to show the behavioral trends of real patients, such as increased collision frequency, decreased walking speed or increased step length variability, and increased head scanning amplitude, compared with the simple blurring method; compared with the simple occlusion method, it can reduce the overcompensation or extreme bias caused by the visible black spot boundary. The above comparison is used to illustrate the rationality and evaluation applicability of the scheme of the present invention at the behavioral level, and the specific numerical results are preferably further verified by controlled experiments.

[0096] like Figure 8 As shown, this embodiment of the invention provides a virtual pre-evaluation system for visual aids based on spatiotemporal pathological features, including: a parameter generation and scene construction module, a visual aid simulation module, a pathological visual simulation module, and an evaluation module; The parameter generation and scene construction module is used to obtain the pathological parameters of the object to be evaluated, and construct a virtual scene in a virtual reality environment based on the pathological parameters to generate sensitivity mask, frequency band loss coefficient and time contrast sensitivity model parameters corresponding to the pathological parameters. The visual assistive device simulation module is used to sequentially perform field cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling on the input image of the virtual scene through the constructed visual assistive device model when the visual assistive device model is enabled, so as to obtain the image to be processed. The pathological visual simulation module is used to perform pathological visual perception processing on the image to be processed in order to obtain the target output image. The evaluation module is used to perform a virtual task test on the target output image and collect the task execution results to output an evaluation result of the visual aid device's assistive effect.

[0097] In practical applications, the parameter generation and scene construction module, the visual aid device simulation module, the pathological visual simulation module, and the evaluation module work together: the parameter generation and scene construction module is responsible for generating individualized pathological parameters, namely sensitivity masks, frequency band loss coefficients, and temporal contrast sensitivity model parameters; the visual aid device simulation module is responsible for performing field cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling on the input image of the virtual scene through the constructed visual aid device model when the visual aid device model is enabled, to obtain the image to be processed, and then transmit the image to be processed to the pathological visual simulation module; the pathological visual simulation module is responsible for performing pathological visual perception processing on the image to be processed to obtain the target output image, and then transmitting the target output image to the evaluation module; the evaluation module is responsible for obtaining the behavioral results corresponding to the target output image under the same task conditions, thereby realizing the virtual pre-evaluation of the assistive effect of the visual aid device.

[0098] Furthermore, embodiments of this application also disclose an electronic device, Figure 9 This is a structural diagram of an electronic device according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0099] Figure 9 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 20 specifically includes: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the virtual pre-assessment method for visual aids based on spatiotemporal pathological features disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment can specifically be a computer.

[0100] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a virtual pre-evaluation channel for visual aids based on spatiotemporal pathological features between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0101] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0102] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform operations and processing on the data 223 in the memory 22. It can be Windows Server, Netware, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the virtual pre-assessment method for visual aids based on spatiotemporal pathological features as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the virtual pre-assessment device for visual aids based on spatiotemporal pathological features from external devices, as well as data collected by its own input / output interface 25.

[0103] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0104] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed virtual pre-evaluation method for visual aids based on spatiotemporal pathological features. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0105] It should be understood that the use of terms such as "method," "apparatus," "unit," and / or "module" in this application is merely to distinguish one method of different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0106] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "a," and / or "the" are not specifically singular and may include the plural. Generally, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements. An element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0107] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0108] If a flowchart is used in this application, it is used to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0109] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A virtual pre-assessment method for visual aids based on spatiotemporal pathological features, characterized in that, Includes the following steps: The pathological parameters of the object to be evaluated are obtained, and a virtual scene is constructed in a virtual reality environment based on the pathological parameters to generate a sensitivity mask, frequency band loss coefficient and time contrast sensitivity model parameters corresponding to the pathological parameters. When the visual assistive device model is enabled, the constructed visual assistive device model sequentially performs field cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling on the input image of the virtual scene to obtain the image to be processed. The image to be processed is subjected to pathological visual perception processing to obtain the target output image; The target output image is subjected to a virtual task test, and the task execution results are collected to output an evaluation result of the visual aid device's assistive effect.

2. The virtual pre-evaluation method for visual aids based on spatiotemporal pathological features according to claim 1, characterized in that, The process involves sequentially performing field-of-view cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling on the input image of the virtual scene using a constructed visual assistive device model to obtain the image to be processed. Specifically, this includes: Perform field-of-view cropping and resampling on the input image; Gamma correction is performed on the cropped and resampled image to enable dynamic range mapping; Perform CLAHE local contrast enhancement on the luminance channel of the dynamically range-mapped image; Edge enhancement is achieved by using the Laplacian operator or Gaussian smoothing difference to sharpen the image after local contrast enhancement. The image with enhanced edges is modeled with system latency in milliseconds by caching historical frames using a circular buffer; and the output update is limited to a preset refresh rate, sampled at the refresh time and displayed using frame hold, to perform refresh rate limitation modeling, so as to obtain the image to be processed.

3. The virtual pre-evaluation method for visual aids based on spatiotemporal pathological features according to claim 1, characterized in that, The step of performing pathological visual perception processing on the image to be processed to obtain a target output image specifically includes: Based on the sensitivity mask and the frequency band loss coefficient, multi-channel Mipmap sampling is performed on the image to be processed to construct low-frequency, mid-frequency and high-frequency channels, and the spatial frequency degraded visual image is reconstructed according to preset or dynamically determined weights. The retinal slip angular velocity and temporal frequency are calculated based on key information of objects in the scene. The pathological weights are generated by combining the retinal slip angular velocity and temporal frequency with the parameters of the temporal contrast sensitivity model. The pathological weights are then used to modulate temporal integration, contrast compression, and saliency attenuation. For visually defective regions, low-frequency texture sampling is performed based on the defect boundary and smoothed sampling coordinates to obtain the target visual fill color, and the target visual fill color is written into the frame buffer to obtain the target output image.

4. The virtual pre-evaluation method for visual aids based on spatiotemporal pathological features according to claim 3, characterized in that, The process of performing multi-channel Mipmap sampling on the image to be processed based on the sensitivity mask and the frequency band loss coefficient, constructing low-frequency, mid-frequency, and high-frequency channels, and reconstructing the spatially frequency-degraded visual image according to preset or dynamically determined weights, specifically includes: The sensitivity mask is sampled based on the texture coordinates of the current pixel to obtain the local visual sensitivity; The target blurring degree of the low-frequency channel, mid-frequency channel and high-frequency channel is calculated based on the local visual sensitivity and the frequency band loss coefficient, respectively. The scene texture is sampled three times in parallel according to the target blur level to obtain the color components of the low-frequency channel, the mid-frequency channel and the high-frequency channel; The visual image is obtained by linearly superimposing the color components and preset or dynamically determined weights.

5. The virtual pre-evaluation method for visual aids based on spatiotemporal pathological features according to claim 4, characterized in that, The preset or dynamically determined weights are jointly determined by local visual sensitivity and low-frequency loss coefficient, mid-frequency loss coefficient, and high-frequency loss coefficient.

6. The virtual pre-evaluation method for visual aids based on spatiotemporal pathological features according to claim 3, characterized in that, The calculation of retinal slip angular velocity and temporal frequency based on key information of scene objects, the generation of pathological weights by combining the retinal slip angular velocity and temporal frequency with the parameters of the temporal contrast sensitivity model, and the modulation of temporal integration, contrast compression, and significance attenuation using the pathological weights, specifically includes: Input the world coordinate velocity, distance and geometric boundary information of objects in the scene, the user's gaze direction, the observer, the camera pose information and preset parameter data; The world coordinate velocity is combined with the observer's or the camera's pose information and mapped onto the retinal coordinate system to obtain the projected angular velocity. The time frequency is calculated based on the projected angular velocity and the target's local spatial frequency characteristics; The time sensitivity ratio is calculated based on the individualized damage coefficient and the time-comparison sensitivity model. The time sensitivity ratio is subjected to nonlinear gating mapping to generate normalized pathological weights; The time integration, contrast compression, and significance attenuation are modulated according to the pathological weights; The key information includes: world coordinate velocity, distance and geometric boundary information, user's gaze direction, observer, camera pose information, and preset parameter data.

7. The virtual pre-evaluation method for visual aids based on spatiotemporal pathological features according to claim 3, characterized in that, For the visually defective region, low-frequency texture sampling is performed based on the defect boundary and smoothed sampling coordinates to obtain the target visual fill color, and the target visual fill color is written into the frame buffer to obtain the target output image. Specifically, this includes: Define the visual defect region and calculate the defect boundary based on the defect mask; Boundary anchor points are determined based on the defective boundary, and smoothed sampling coordinates are calculated based on a nonlinear weighting function; Low-frequency texture sampling is performed based on the smoothed sampling coordinates to obtain the target visual fill color of the defective area; The target visual fill color is written to the frame buffer to obtain the target output image.

8. The virtual pre-evaluation method for visual aids based on spatiotemporal pathological features according to claim 7, characterized in that, The visual defect area is generated by mapping the clinical visual field sensitivity distribution to visual field angle coordinates from screen pixels and thresholding, and the temporal and nasal directions are corrected according to the relationship between the left and right eyeglass images.

9. The virtual pre-evaluation method for visual aids based on spatiotemporal pathological features according to claim 1, characterized in that, The pathological parameters include at least clinical visual field sensitivity distribution parameters, contrast sensitivity-related parameters, and time-contrast sensitivity-related parameters, wherein: The clinical visual field sensitivity distribution parameters are used to generate the sensitivity mask or the visual defect region; The contrast sensitivity related parameters are used to generate low-frequency loss coefficients, mid-frequency loss coefficients, and high-frequency loss coefficients; The time-comparison sensitivity parameters are used to generate individualized damage coefficients or time-comparison sensitivity model parameters.

10. A virtual pre-assessment system for visual aids based on spatiotemporal pathological features, characterized in that, include: The parameter generation and scene construction module is used to obtain the pathological parameters of the object to be evaluated, and construct a virtual scene in a virtual reality environment based on the pathological parameters to generate sensitivity mask, frequency band loss coefficient and time contrast sensitivity model parameters corresponding to the pathological parameters. The visual assistive device simulation module is used to sequentially perform field cropping and resampling, dynamic range mapping, local contrast enhancement, edge enhancement, and temporal resampling on the input image of the virtual scene through the constructed visual assistive device model when the visual assistive device model is enabled, so as to obtain the image to be processed. The pathological visual simulation module is used to perform pathological visual perception processing on the image to be processed in order to obtain the target output image. The evaluation module is used to perform virtual task tests on the target output image and collect the task execution results to output the evaluation results of the visual aid device's assistive effect.

Images