A visual perception enhancement method based on cross-modal cue reactivation

Abstract

The present application belongs to the technical field of visual perception enhancement, and is a visual perception enhancement method based on cross-modal cue reactivation, comprising the following steps: establishing a cross-modal correspondence between target visual features and candidate sound cues to form a candidate cross-modal mapping record to be verified; performing a cross-modal association verification task, recording verified cross-modal mappings, and marking associated cues; obtaining the visual discrimination threshold of the target visual features before reactivation; selecting a reactivation path according to the user state, reactivating the corresponding visual representation in the verified cross-modal mapping using the associated cues; calculating the visual performance improvement based on the visual discrimination threshold before reactivation and the visual discrimination threshold after reactivation; and updating and optimizing the reactivation strategy according to the visual performance improvement and the visual enhancement efficiency per unit effective trial. The present application can solve the problem that traditional wakeful visual training continuously occupies the visual channel and has a high burden of active participation, and cannot act on low-level visual features in a targeted manner.

Description

Technical Field

[0001] This invention relates to the field of visual perception enhancement technology, and more specifically to a visual perception enhancement method based on cross-modal cue reactivation. Background Technology

[0002] Visual perception learning typically relies on numerous, repetitive waking visual tasks. Existing visual perception learning methods generally present gratings, moving dots, characters, or other visual stimuli continuously, requiring users to continuously gaze, identify, and press keys. The difficulty of the stimuli is adjusted based on the response results, and visual performance is assessed by comparing accuracy or discrimination thresholds before and after training. This type of approach continuously occupies the visual channel and places demands on attention maintenance, task comprehension, hand manipulation, and training compliance, resulting in a high burden of active participation. For individuals who have difficulty cooperating with training for extended periods, the training dosage and completion rate are easily limited. On the other hand, target memory reactivation technology can replay sensory cues that appeared during the learning phase during wakefulness or sleep to influence existing memories. Existing technologies reconstruct the sensory environment of waking learning during slow-wave sleep to enhance knowledge consolidation or use EEG to determine sleep stages and neural landmarks before outputting intervention signals. These primarily target declarative knowledge, contextual content, or general memory enhancement, and do not establish a directional mapping, association quality gating, or feature-specific visual threshold evaluation chain for low-level visual orientation and retinal position.

[0003] Existing visual training methods can generate individualized training tasks using parameters such as sinusoidal gratings, Gabor gratings, orientation, position, contrast, or spatial frequency. However, these methods still primarily rely on the repeated presentation of visual stimuli, with sound mainly used for feedback, prompts, or to increase engagement. They do not utilize auditory cues as control signals for accessing encoded visual features in the absence of target visual input. The use of cross-modal cues also carries directional risks; unverified sounds, sounds that do not correspond to the target visual features, or sounds output under inappropriate mental states may not necessarily lead to improvement and could even cause a decline in target visual performance. If only whether sound is output is recorded, without recording the mapping quality between sound and visual features, the user's current mental state, abnormal arousal, and the correspondence between features before and after tests, it is difficult to determine whether a reactivation truly affects the intended visual representation. Summary of the Invention

[0004] To address the technical problems existing in the prior art, this invention provides a visual perception enhancement method and system based on cross-modal cue reactivation, which can solve the problems of traditional awake visual training continuously occupying the visual channel, having a high burden of active participation, and the inability of general memory reactivation schemes to target low-level visual features and lack of a unified evaluation link.

[0005] The objective of this invention can be achieved by adopting the following technical solutions:

[0006] A visual perception enhancement method based on cross-modal cue reactivation includes the following steps:

[0007] S1. Under the condition that the target visual stimulus and candidate sound cues are presented simultaneously, perform association learning to establish the cross-modal correspondence between the target visual features and the candidate sound cues, and form a candidate cross-modal mapping record to be verified.

[0008] S2. Perform a cross-modal association verification task. Calculate the association quality of candidate cross-modal mapping records based on the user's response to the cross-modal association verification task. When the association quality reaches a preset value, store the candidate cross-modal mapping records as verified cross-modal mappings and mark the candidate audio cues as associated cues.

[0009] S3. Perform a silent adaptive visual discrimination task to obtain the visual discrimination threshold of the target visual features before reactivation.

[0010] S4. Collect the user's EEG and EEG signals, determine whether the user is in a stable N3 sleep state or in a waking state based on the user's EEG and EEG signals, and select a reactivation path according to the user's state.

[0011] S5. Without presenting the target visual stimulus, reactivate the corresponding visual representation in the validated cross-modal mapping by using pre-associated cues that are associated with the target visual features.

[0012] S6. After the reactivation phase ends, calculate the visual discrimination threshold after reactivation, and calculate the improvement in visual performance based on the visual discrimination threshold before reactivation and the visual discrimination threshold after reactivation.

[0013] S7. Calculate the number of effective reactivation trials and the visual enhancement efficiency per effective trial. If the improvement in visual performance or the visual enhancement efficiency per effective trial does not meet the preset conditions, return to step S5 to update and optimize the reactivation strategy.

[0014] Specifically, step S1 includes:

[0015] S11. Define the target visual features that need to be enhanced, and define the candidate audio cue parameters;

[0016] S12. Control the visual presentation terminal to present visual stimuli with target visual features, and control the sound cue output unit to output candidate sound cues synchronized with the visual stimuli, perform visual orientation discrimination task and respond through the response input unit, and establish cross-modal correspondence between target visual features and candidate sound cues.

[0017] S13. Based on the cross-modal correspondence between the target visual features and the candidate audio cues, establish a candidate cross-modal mapping record.

[0018] Specifically, step S2 includes:

[0019] S21. Perform cross-modal association verification task, control the sound cue output unit to output sound stimuli composed of learned reference tones, control the visual presentation terminal to present the target grating and interference grating in the left and right visual fields respectively, and the user determines whether the target grating is located in the left or right visual field and responds through the response input unit.

[0020] S22. Calculate the number of correct responses based on the user's response to the cross-modal association verification task, and calculate the association quality based on the number of correct responses.

[0021] Specifically, step S3 includes:

[0022] S31. Establish an initial threshold probability distribution, use the QUEST adaptive ladder method to control the visual presentation terminal to present two gratings in sequence, and record the user's response result in the i-th trial; select the orientation difference as the visual discrimination threshold based on the current posterior distribution.

[0023] S32. Update the threshold posterior distribution. Determine the likelihood of the response result of the i-th trial under the conditions of candidate threshold and orientation difference based on the psychometric function, and then update the threshold posterior distribution according to Bayes' rule.

[0024] S32. After completing the preset number of trials, estimate the visual discrimination threshold of a single QUEST adaptive ladder based on the final posterior distribution.

[0025] Specifically, step S4 includes:

[0026] S41. The user's brainwave and electrooculogram (EOG) signals are continuously acquired through the EEG and EOG acquisition unit, and the brainwave and EOG signals are divided according to a preset time window.

[0027] S42. Based on the user's EEG and EOG signals, determine whether the user has entered a stable N3 sleep state and whether the user is awake.

[0028] S43. When the user has entered a stable N3 sleep state, and the EEG and EEG signals are continuous and recorded completely, without any obvious artifacts or signal interruptions that affect the determination of sleep state, select the sleep reactivation path; when the user is awake, sleep monitoring conditions are not available, or reactivation needs to be completed immediately, select the awake reactivation path.

[0029] Specifically, step S5 includes:

[0030] S51. When selecting the wake-up reactivation path, complete the sound discrimination task and output multiple wake-up reactivation sound trials. Each wake-up reactivation sound trial outputs two sound stimuli in sequence, without presenting visual stimuli containing the target orientation and target location, and determine the sound perception threshold and safe output upper limit.

[0031] S52. When selecting a sleep reactivation path, based on the associated cues in the verified cross-modal mapping, set multiple sound reactivation trials into a preset short sleep sound sequence and output the short sleep sound sequence.

[0032] Specifically, step S7 includes:

[0033] S71. Set a valid flag for each reactivation attempt and count the number of valid reactivation attempts;

[0034] S72. Calculate the visual enhancement efficiency per unit of effective reactivation trials based on the effective number of reactivation trials and the percentage improvement in visual performance.

[0035] S73. If the visual performance improvement amount and the visual enhancement efficiency per unit effective trial do not meet the preset conditions, return to step S5 and adjust the reactivation path, associated cues, output dose, and state gating parameters. When the visual performance improvement amount (VPI) reaches the preset statistical criterion and the visual enhancement efficiency per unit effective trial increases by a preset amount compared to the visual enhancement efficiency per unit effective trial before adjustment, it is considered that the current reactivation parameters simultaneously meet the requirements of visual enhancement effect and reactivation efficiency, and the strategy for optimizing the reactivation process is stopped.

[0036] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0037] This invention provides a visual perception enhancement method based on cross-modal cue reactivation. By treating visual feature encoding, auditory cue parameters, and association quality as a single mapping record, auditory reactivation is explicitly directed to a predetermined orientation and visual field location, rather than general memory content. During the sleep reactivation phase, the target visual stimulus is not presented again, nor is visual discrimination, auditory judgment, or key press response required, thus reducing the burden of active participation caused by continuously occupying the visual channel and maintaining attention and active operation.

[0038] The visual enhancement efficiency per effective trial is obtained by dividing the improvement in visual performance by the number of effective trials. Trial-normalized comparisons can be made between sleep reactivation, wakefulness reactivation, and visual training at different trial counts. The visual enhancement efficiency per effective trial for sleep reactivation is significantly higher than that for wakefulness reactivation and dose-matched visual training. During wakefulness, sound perception threshold gating is used, while during sleep, N3 gating, short sequence output, and abnormal pauses are used to avoid applying the same sound strategy indiscriminately to different brain states. Both wakefulness and sleep reactivation can lead to target-specific improvements. The wakefulness pathway can be used when sleep monitoring is not required or when immediate training is needed, while the sleep pathway further improves the efficiency per trial and reduces user intervention. Through associated validation and exclusion of unassociated cues, the risk of erroneous sound mapping leading to a decline in target visual performance during sleep is reduced. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram of the overall process of the visual perception enhancement method based on cross-modal cue reactivation in an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the overall process of learning the association between visual features and auditory cues in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of a single trial of visual-sound association learning in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of a single trial of a cross-modal correlation verification task in an embodiment of the present invention;

[0044] Figure 5 This is a schematic diagram of a single trial of cross-modal cue reactivation in a conscious state in an embodiment of the present invention;

[0045] Figure 6 This is a schematic diagram of the cross-modal cue reactivation process in a stable N3 sleep state according to an embodiment of the present invention;

[0046] Figure 7 This is a schematic diagram of visual discrimination threshold measurement and visual feature specificity evaluation before and after reactivation in an embodiment of the present invention;

[0047] Figure 8This is a schematic diagram comparing the visual enhancement efficiency per unit of effective trials between sleep reactivation and matched wakeful visual training in an embodiment of the present invention.

[0048] Figure 9 This is a schematic diagram comparing the visual enhancement efficiency per unit of effective trials for sleep reactivation and wakefulness reactivation in an embodiment of the present invention. Detailed Implementation

[0049] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments, and the implementation of the present invention is not limited thereto. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In the text, "associated cues" refers to sound cues that have passed association verification and reached the preset association criteria; "unassociated cues" refers to sound cues that have not been learned synchronously with the target visual features or have not reached the association criteria; "reactivation stage" refers to the stage in which associated cues are output without presenting the target visual stimulus again; "stable N3" refers to a slow-wave sleep state that can continuously output short sequences of sleep based on the current EEG, EEG, and continuous state information.

[0050] Example 1:

[0051] like Figure 1 The diagram shows an overall flowchart of a visual perception enhancement method based on cross-modal cue reactivation. This embodiment of the visual perception enhancement method based on cross-modal cue reactivation includes:

[0052] S1. Establish candidate cross-modal mappings. Under the condition that the target visual stimulus and candidate sound cues are presented synchronously, perform association learning to establish the cross-modal correspondence between the target visual features and the candidate sound cues, and form a candidate cross-modal mapping record to be verified.

[0053] S11. Define the target visual features that need to be enhanced, and define the candidate audio cue parameters.

[0054] Specifically, the target visual features that need to be enhanced include: the orientation of the visual stimulus, the visual field location or coordinates of the visual stimulus, C representing the contrast of the visual stimulus, the spatial frequency of the visual stimulus, and one or more of the motion direction or motion coherence. The target visual feature vector can be represented as:

[0055] ;

[0056] Where F represents the target visual feature vector, θ represents the orientation of the visual stimulus, P represents the visual stimulus's visual field position or position coordinates, C represents the visual stimulus's contrast, SF represents the visual stimulus's spatial frequency, and D represents the motion direction or motion coherence.

[0057] In this embodiment, the target visual features that need to be enhanced include at least orientation. and field of view position Determine the target's visual features; for example, the target's visual features can be represented by the target's orientation. With the target location The combination of these parameters. The acoustic parameters of a candidate acoustic cue include: the reference frequency of the sound, the sound pressure level of the sound, the duration of a single acoustic stimulus, the time interval between adjacent acoustic stimuli, and one or more sound output patterns. The acoustic parameters of a candidate acoustic cue are expressed as:

[0058] ;

[0059] in, Audio parameters representing candidate audio cues. Represents the reference frequency of sound. Indicates the sound pressure level. Indicates the duration of a single sound stimulus. Indicates the time interval between adjacent sound stimuli. This indicates the sound output mode. The sound output mode can be a single tone, a repeating tone, a tone pair, or a short sequence of multiple sounds.

[0060] In this embodiment, two reference tones, 320 Hz and 860 Hz, can be used as candidate sound cues. The correspondence between the two reference tones and the target visual features is randomly assigned between different users or between different task rounds to reduce the influence of fixed sound frequencies on the results.

[0061] S12. Control the visual presentation terminal to present visual stimuli with target visual features, and control the sound cue output unit to output candidate sound cues synchronized with the visual stimuli. Perform the visual orientation recognition task and respond through the response input unit to establish a cross-modal correspondence between target visual features and candidate sound cues.

[0062] Specifically, the visual presentation terminal is used to display the gaze point, raster stimuli, selection interface, and feedback information. The visual presentation terminal can be a fixed display terminal, projection terminal, virtual display terminal, or near-eye display terminal. The visual presentation terminal should be calibrated for display size, viewing distance, brightness, contrast, and viewing angle. The sound output unit is used to output candidate sound cues, associated cues, unassociated comparison cues, sound discrimination stimuli, and background white noise. The sound output unit can include headphones, speakers, or other sound playback devices calibrated for sound pressure level. The response input unit is used to receive a binary response from the user. A binary response means that the user selects one option from two mutually exclusive options, such as selecting whether the second raster rotates clockwise or counterclockwise relative to the first raster in a visual orientation discrimination task. The response input unit can be a button box, keyboard, touchscreen, eye-tracking selection device, or other input device capable of generating a binary result.

[0063] In this embodiment, the target visual features include the orientation and visual field position of the visual stimulus. The visual stimulus is at least one of a sinusoidal grating, a Gabor grating, a square wave grating, a moving dot matrix, a character, or a natural image. The user completes a visual feature-audio cue association learning task, establishing a cross-modal correspondence between the target visual features and candidate audio cues. This cross-modal correspondence means that, within the same user and the same task round, a target visual feature vector F is uniquely assigned to a candidate audio cue, and in multiple association learning trials, the visual stimulus containing the target visual feature and the corresponding candidate audio cue appear synchronously.

[0064] like Figure 2-3 As shown, Figure 2 This is a schematic diagram of the overall process of learning the association between visual features and auditory cues. The target visual features and candidate auditory parameters are simultaneously encoded and associated to form a candidate cross-modal mapping. In this embodiment, a visual orientation discrimination task is performed. The user determines whether the second grating is rotated clockwise or counterclockwise relative to the first grating and responds clockwise or counterclockwise. Specifically, the visual presentation terminal presents grating stimuli after viewpoint calibration. The viewing distance can be set to 57 cm, or an equivalent viewpoint calibration can be performed based on the terminal size and actual viewing distance. Specifically, the target orientation... The target location for the association learning phase is determined within the range of 0° to 180°. The eccentricity can be set to approximately 3.5° in the lower left or upper right visual quadrant. During the association verification phase and in the pre- and post-visual tests, the eccentricity of the left and right visual stimuli can be set to approximately 4.5°. Within the same task round, the corresponding stimulus position parameters remain fixed. For example... Figure 3 The diagram illustrates a single trial of visual-audio association learning. A single trial involves two sequentially presented gratings. The orientation of the first grating is... The orientation of the second grating is or Each grating lasts for 200 ms, with a 600 ms gap between the two gratings. When both gratings are presented, candidate auditory cues already assigned to the target visual features are output synchronously. The user judges whether the second grating rotates clockwise or counterclockwise relative to the first grating. The user's judgment of whether the second grating rotates clockwise or counterclockwise relative to the first grating is used with an adaptive ladder (QUEST, Adaptive Sychophysical Method). Based on the user's correct or incorrect answer, the orientation difference used in the next trial is adjusted to maintain the preset difficulty of the association learning task and sustain the user's processing of the synchronously presented visual stimuli and candidate auditory cues. In this embodiment, the association learning phase may include 4 sets of trials, each containing 40 trials. The QUEST in this step is used to adaptively control the visual discrimination difficulty of the association learning phase; the obtained threshold estimate is not used as the visual discrimination threshold before reactivation. The visual discrimination threshold before reactivation is measured in step S3 under no-sound conditions for both the target condition and the transfer test condition.

[0065] S13. Based on the cross-modal correspondence between the target visual features and the candidate sound cues, establish candidate cross-modal mapping records to be verified. A cross-modal mapping database can be constructed based on the candidate cross-modal mapping records.

[0066] In the association learning process of step S1, candidate cross-modal mapping records to be verified are first established. The cross-modal mapping database is established during the execution of steps S1 and S2. A cross-modal mapping record in the database is represented as:

[0067] ;

[0068] in, This represents a cross-modal mapping record. Identify the visual features of the target. The audio parameters for candidate audio cues. For related quality, To map version numbers, To map the timestamps that have been verified and written to the database, The mapping is in a valid state. The version number is [not specified] when the mapping is first validated. Set to 1. When the target visual features, sound frequency, sound pressure level, sound output mode, or associated verification results change, a new mapping record is created, and the version number is incremented.

[0069] In the subsequent step S2, when the association quality reaches a preset value, and the mapping record has not expired, has not been manually revoked, and there are no audio parameter conflicts, the setting is... When the association quality fails to meet the criteria, the mapping expires, the mapping is revoked, or there are conflicting sound parameters, the settings are adjusted accordingly. The reactivation phase only calls... Furthermore, the version number is the latest version of the mapping record, and the target visual feature identifier, sound parameters, mapping version number and output timestamp are written into each sound output log.

[0070] S2. Perform a cross-modal association verification task. Calculate the association quality of candidate cross-modal mapping records based on the user's response to the cross-modal association verification task. When the association quality reaches a preset value, store the candidate cross-modal mapping records as verified cross-modal mappings and mark the candidate audio cues as associated cues.

[0071] S21. Perform cross-modal association verification task, control the sound cue output unit to output sound stimuli composed of learned reference tones, control the visual presentation terminal to present the target grating and interference grating in the left and right visual fields respectively, and the user judges whether the target grating is located in the left or right visual field and responds through the response input unit.

[0072] like Figure 4 As shown in the diagram, this is a single-trial illustration of a cross-modal association verification task. In each trial, the sound cue output unit outputs a sound stimulus composed of a learned reference tone. The sound stimulus may include two sequentially output sounds, each lasting 200 ms, with a 600 ms interval between the two sounds. Subsequently, the visual presentation terminal displays a target grating and an interference grating in the left and right visual fields, respectively. The target grating includes the target orientation. The orientation of the interference grating is deflected by a preset angle relative to the target orientation, for example, by 60°. Whether the target grating is located on the left or right side is randomly determined in different trials. The user determines whether the target grating is located in the left or right field of view, and the response input unit responds with either left or right. The correctness of the response is determined based on the actual position of the target grating.

[0073] S22. Calculate the number of correct responses based on the user's response to the cross-modal association verification task, and calculate the association quality Q based on the number of correct responses. The association quality Q is used to determine whether the user has formed a reliable correspondence between the target visual features and the auditory cues.

[0074] The association quality Q in step S2 is calculated based on the left or right position judgment result in the association verification stage. The clockwise or counterclockwise response obtained in the association learning stage of step S1 is used for QUEST adaptive ladder update and maintaining audiovisual pair learning, and is not included in the calculation of association quality Q. The association verification in this embodiment may include 3 sets of trials, 60 trials in each set, for a total of 180 verification trials. Assume that the association verification task includes a total of There are 10 valid trials, of which 100 have correct responses. Then the matching accuracy for:

[0075] ;

[0076] in, Indicates the matching accuracy. Indicates the number of correct responses. Indicates the number of valid verification tests.

[0077] To simultaneously consider matching accuracy and the number of validation trials, this embodiment preferably uses the lower limit of the 95% Wilson confidence interval for matching accuracy as the association quality. Related quality Represented as:

[0078] ;

[0079] in, This represents the lower bound of the 95% confidence interval for matching accuracy. Indicates the matching accuracy. Indicates the number of correct responses. Indicates the number of valid verification tests. This represents the critical value at the corresponding confidence level in a standard normal distribution. At a 95% confidence level, Using the lower bound of the confidence interval instead of just the matching accuracy can reduce the risk of accidental high accuracy leading to incorrect mappings when the number of validation trials is small. The chance level is 0.5 higher than that of a two-choice task, and the matching accuracy is... Achieve the preset minimum accuracy When this happens, the cross-modal mapping is determined to have passed verification.

[0080] In other implementations, signal detection theory indicators may also be used. Evaluate the correlation quality. When the target being located on a preset side is defined as a signal, Calculate according to the following formula:

[0081] ;

[0082] in, Indicates hit rate. Indicates the false alarm rate. This represents the inverse function of the cumulative distribution function of the standard normal distribution. In the same embodiment, the calculation method for correlation quality should be fixed in advance to avoid arbitrarily changing evaluation indicators in different users or different task rounds.

[0083] When the association quality meets the preset association criteria, the target visual features will be... Sound parameters Related quality Mapping version Establishment time and effective state The information is written to the cross-modal mapping database, and the corresponding audio cue is marked as an associated cue. If the association quality does not meet the preset association criteria, the audio cue is not allowed to enter the enhancement sequence. The system can re-execute association learning, replace candidate audio cues, or terminate the current mapping process. Another benchmark sound that has not been learned synchronously with the target visual stimulus can be marked as an unassociated cue. Unassociated cues are only used for comparative experiments or quality verification and are not included in reactivation sequences aimed at improving visual performance by default.

[0084] S3. Verify and confirm the associated cues, perform a silent adaptive visual discrimination task, and obtain the visual discrimination threshold of the target visual features before reactivation.

[0085] Steps S1 and S3 can employ the same visual discrimination procedure and the QUEST adaptive ladder method, but their task objectives and results differ. The adaptive adjustment in step S1 is only used to control the task difficulty during the association learning phase. The silent adaptive visual discrimination task in step S3 refers to controlling the visual presentation terminal to sequentially present two gratings with different orientations without outputting audio cues. The user determines whether the second grating rotates clockwise or counterclockwise relative to the first grating, and adaptively adjusts the orientation difference based on the response to obtain the pre-reactivation visual discrimination threshold corresponding to 75% accuracy. This pre-reactivation visual discrimination threshold serves as a baseline value for comparison with the post-reactivation visual discrimination threshold.

[0086] S31. Establish an initial threshold probability distribution. Using the QUEST adaptive ladder method, control the visual presentation terminal to sequentially present two gratings and record the user's response in the i-th trial. Select the orientation difference as the visual discrimination threshold based on the current posterior distribution. The initial threshold probability distribution is expressed as:

[0087] ;

[0088] in, This represents the minimum orientation difference required for the user to achieve the preset accuracy rate.

[0089] In this embodiment, the orientation difference corresponding to 75% accuracy in a two-way orientation discrimination task is used as the visual discrimination threshold. The initial probability distribution can be determined based on pre-experiments, previous measurement results, or a preset threshold range. Specifically, in the i-th trial, the QUEST adaptive ladder selects the orientation difference based on the current posterior distribution. The visual display terminal sequentially presents two gratings. The first grating is oriented as follows: The orientation of the second grating is or Each grating lasts for 200 ms, and the interval between two gratings is 600 ms. The user determines whether the second grating rotates clockwise or counterclockwise relative to the first grating. The response results of each trial are denoted as When the user answers correctly, When the user answers incorrectly, .

[0090] S32. Update the posterior distribution of the threshold, and determine the occurrence of the first [condition] under the candidate threshold and orientation difference based on the psychometric function. The likelihood of the response results for each trial is calculated, and then the threshold posterior distribution is updated according to Bayes' rule. Response results The likelihood representation is:

[0091] ;

[0092] The posterior distribution of the Bayesian rule-updated threshold is represented as follows:

[0093] ;

[0094] in, Indicates the first The threshold posterior distribution before each trial begins Indicates obtaining the first The threshold posterior distribution after each trial response is used, and the denominator is used to normalize the updated posterior distribution. The QUEST ladder selects the orientation difference for the next trial based on the updated posterior distribution. When the user answers correctly, the next trial typically uses a smaller orientation difference; when the user answers incorrectly, the next trial typically uses a larger orientation difference.

[0095] S33. After completing the preset number of trials, based on the final posterior distribution... Estimate the visual discrimination threshold of a single QUEST step, where This indicates the number of trials required to complete the ladder.

[0096] The posterior mean is calculated using the following formula:

[0097] ;

[0098] Posterior median satisfy:

[0099] ;

[0100] The posterior mode is determined by the following formula:

[0101] ;

[0102] In this embodiment, the posterior mean is preferably used. As a threshold estimate for a single QUEST ladder, four adaptive QUEST ladders are set for each test condition in the pre-test, and the threshold estimates of the four ladders are taken as an arithmetic mean to obtain the visual discrimination threshold before reactivation corresponding to 75% accuracy for that condition. The obtained visual discrimination threshold before reactivation is used as the baseline value for calculating the improvement in visual performance in step S6.

[0103] S4. Collect the user's EEG and EEG signals, determine whether the user is in a stable N3 sleep state or in a waking state based on the user's EEG and EEG signals, and select a reactivation path according to the user's state.

[0104] S41. Acquire and segment physiological signals. Continuously acquire the user's EEG and EEG signals through the EEG and EEG acquisition units, and divide the EEG and EEG signals into preset time windows.

[0105] Specifically, the EEG and EEG acquisition unit continuously acquires the user's EEG and EEG signals and sends them with a unified timestamp. The function of the EEG and EEG acquisition unit is to provide user state determination and security gating basis for the sleep path. Physiological signals are divided into preset time windows; this embodiment can use a 30-second time window, or a shorter time window for real-time state updates. For each time window, awake (W), N1, N2, N3, or REM sleep labels are output. Brain state determination can be completed by preset rules, a machine learning classifier, manual confirmation, or a combination of algorithms and manual intervention.

[0106] S42. Based on the user's EEG and EOG signals, determine whether the user has entered a stable N3 sleep state and whether the user is awake.

[0107] Specifically, based on slow-wave activity, sleep spindle activity, and their combinations in EEG signals, as well as EEG signals, the user's brain state is identified, including: wakefulness, N1 sleep, N2 sleep, N3 sleep, REM sleep, arousal response, body movement, and signal quality abnormalities, to determine whether the user has entered a stable N3 sleep state. N3 sleep is a deep non-REM sleep stage, characterized primarily by high-amplitude slow-wave activity in the range of 0.5 Hz to 2 Hz.

[0108] A stable N3 sleep state refers to a state in which the current time window and at least one adjacent time window continuously meet the N3 criterion, or the N3 state is maintained continuously for a preset duration. Only when a stable N3 sleep state is consistently achieved... Only after this process is complete is the short sleep sound sequence allowed to be initiated. This reduces the likelihood of sound cues being output during sleep stage boundaries, brief awakenings, or periods of signal instability. A 30-second sleep stage window can be used, and two consecutive 30-second windows being classified as N3, with no detected awakenings or significant body movements and physiological signal quality meeting preset requirements, serves as one of the initiation conditions for a stable N3 state. In this implementation, the duration of the stable state is no less than 60 seconds. Alternatively, a 10-second or other shorter sliding window can be used for real-time status updates. Regardless of the time window used, the short sound sequence is only initiated if the N3 and signal quality requirements are consistently met within the most recent consecutive period. The determination of N3 can be implemented using an online automatic algorithm conforming to sleep stage standards, manual stage determination, or a combination of both.

[0109] In one embodiment, a user is determined to have entered a stable N3 sleep state when the EEG and EEG signals of a time window simultaneously meet the following conditions: 1. Slow wave activity in the EEG signal reaches a preset proportion or a preset power threshold; 2. The EEG signal does not show rapid eye movement characteristics; 3. No arousal response or obvious body movement is detected; 4. The quality of physiological signals reaches a preset threshold; 5. The sleep stage classification result is N3.

[0110] A state of wakefulness can be determined through one or more of the following: 1. The user actively selects a wakefulness working mode. 2. The brain state determination module outputs a wakefulness (W) label. 3. The user is able to continuously complete a wakefulness sound recognition task. 4. Electrooculography (EOG), electroencephalography (EEG), or body movement information matches the characteristics of a wakefulness state. When the user selects a wakefulness working mode and the state monitoring does not show sleep or abnormal signals, the wakefulness reactivation path is allowed to be initiated.

[0111] S43. Select a reactivation path. The reactivation path includes a sleep reactivation path and a wakefulness reactivation path. When the user has entered a stable N3 sleep state and the EEG and EEG signals are continuous and recorded completely, without any obvious artifacts or signal interruptions that affect the determination of sleep state, the sleep reactivation path is selected. When the user is awake, does not currently have the conditions for sleep monitoring, or needs to complete reactivation immediately, the wakefulness reactivation path can be selected.

[0112] When the above signal quality conditions are not met, or when the sound output causes an awakening response, obvious body movement, or a shift in sleep stage, the sleep reactivation pathway is not activated or is suspended.

[0113] S5. Output associated cues and implement state gating. Without presenting the target visual stimulus again, use the associated cues that have been pre-associated with the target visual features to reactivate the corresponding visual representations in the validated cross-modal mapping.

[0114] S51. When selecting the wakefulness-reactivation path, multiple wakefulness-reactivation sound trials are output based on the associated cues in the verified cross-modal mapping. Each wakefulness-reactivation sound trial outputs two sound stimuli in sequence, without presenting visual stimuli containing the target orientation and target location. The user completes the sound discrimination task and determines the sound perception threshold and safe output upper limit.

[0115] The sound-perceptible threshold and safe output upper limit are used to define the allowable output intensity range of the subsequent wake-up reactivation sound, and serve as the conditions for initiating and continuously operating the wake-up reactivation path. The sound-perceptible threshold represents the minimum sound output intensity required for the user to achieve a preset accuracy rate. Wake-up reactivation is only initiated when the sound output intensity is higher than the user's sound-perceptible threshold but does not exceed the safe output upper limit; when the sound output intensity does not meet the above range, the controller either does not initiate wake-up reactivation or pauses the sound output and adjusts the sound intensity.

[0116] The sound perception threshold can be obtained through a binary sound detection task or a sound discrimination task. This threshold represents the minimum sound output intensity required for the user to achieve a preset accuracy rate. The safe output upper limit is preset based on auditory safety requirements, device calibration results, and user tolerance. Awake reactivation only initiates when the sound output intensity exceeds the user's sound perception threshold but does not exceed the safe output upper limit.

[0117] like Figure 5 The diagram illustrates a single trial of cross-modal cue reactivation in a conscious state. In each conscious reactivation sound trial, two sound stimuli are sequentially output through the sound output unit. Each sound lasts for 200 ms, with a 600 ms interval between the two sounds. The second sound varies by 5 Hz relative to the 320 Hz or 860 Hz reference pitch (i.e., increases or decreases by 5 Hz). The user judges whether the second sound is higher or lower in pitch relative to the first sound. The start time interval between adjacent trials varies randomly within the range of 4.0 s to 4.5 s. Feedback can be provided through gaze point changes or other low-interference methods when the user answers incorrectly. This embodiment of conscious reactivation includes 6 groups of sound trials, each group containing 40 trials, for a total of 240 sound reactivation trials.

[0118] During wakefulness-reactivation, no visual stimuli containing target orientation or location are presented; the user performs a sound discrimination task. Wakefulness-reactivation can also employ passive listening, low-load sound discrimination, or other wakefulness tasks that do not involve target visual stimuli. Regardless of the wakefulness task used, the following conditions should be maintained: 1. Use validated and currently effective cross-modal mappings. 2. The strength of the associated cues is higher than the user's sound perception threshold. 3. No visual stimuli containing target visual features are presented during reactivation. 4. Isomorphic aphonic visual threshold measurements are used before and after reactivation.

[0119] Embedding sound into high-intensity background noise and making it unreliable for the user to distinguish the sound can be considered an imperceptible comparison condition. Imperceptible comparison conditions are not considered preferred wakefulness enhancement paths.

[0120] S52. When selecting a sleep reactivation path, based on the associated cues in the verified cross-modal mapping, multiple sound reactivation trials are set into a preset short sleep sound sequence, and the short sleep sound sequence is output. Simultaneously, the user is not required to perform visual discrimination, auditory judgment, or key press responses.

[0121] like Figure 6 The diagram illustrates the cross-modal cue reactivation process during stable N3 sleep. During sleep reactivation, the EEG and EEG acquisition units continuously collect the user's physiological signals. Only after the brain state determination module confirms the user is in stable N3 sleep is a short audio sequence allowed to be output. In this embodiment, 10 audio reactivation trials are set as a preset short sequence. Each trial can use the same audio structure as wakefulness reactivation, but the user does not need to perform visual discrimination, auditory judgment, or key presses during sleep reactivation. Background white noise with a sound pressure level of 35 dB to 45 dB can be continuously output during sleep monitoring and audio reactivation. This embodiment can use approximately 40 dB sound pressure level background white noise to reduce the possibility of awakening due to sudden sound changes. The sound pressure level of the associated cues is determined based on the individual's auditory threshold, background noise intensity, and historical awakening event incidence. The sound pressure level of the associated cues should be lower than the preset awakening upper limit and meet auditory safety requirements. Sleep reactivation only invokes valid states. Furthermore, the version number must be the latest verified mapping. Unassociated cues, invalid mappings, and expired mappings are not included in the sleep enhancement sequence by default. If it is necessary to set comparison conditions for unassociated cues, independent task identifiers, independent sound output time periods, and independent post-test result identifiers should be used to prevent the results of unassociated cues from entering the enhancement path update.

[0122] Specifically, during the output of short sleep sound sequences, if an abnormality is detected, the current short sound sequence is immediately paused; once a stable N3 sleep state is detected again and continues for a preset duration, a new short sound sequence is started. Specifically, abnormalities include one or more of the following: 1. Arousal response or user awakening. 2. Detection of significant body movement. 3. User entering N1 sleep or REM sleep. 4. User continuously leaving a stable N3 sleep state. When pausing the current short sound sequence, the type of abnormal event, the time of occurrence, the number of completed trials, and the current mapping version are recorded. Once a stable N3 sleep state is detected again and continues for a preset duration, a new short sound sequence is started. Trials that were not fully output when an abnormality occurred are not counted in the valid reactivation trial count. If consecutive arousal events occur, the sound intensity can be reduced, the trial interval extended, the stable N3 waiting time increased, or the current reactivation terminated.

[0123] Traditional visual training typically requires repeated presentation of target visual stimuli and demands that users continuously perform visual discrimination and behavioral responses, achieving training results through repeated active practice. Step S5 of this invention eliminates the need for repeated presentation of target visual stimuli. Instead, it utilizes pre-associated auditory cues with target visual features to selectively reactivate the corresponding visual representations, thereby enhancing visual perception. The most prominent feature of this approach is its ability to further promote learned visual perception while reducing repetitive visual stimulation and requiring continuous active engagement.

[0124] S6. After the reactivation phase ends, calculate the visual discrimination threshold after reactivation, and calculate the improvement in visual performance based on the visual discrimination threshold before reactivation and the visual discrimination threshold after reactivation.

[0125] S61. Perform a silent visual post-test task and calculate the visual discrimination threshold after reactivation.

[0126] Specifically, in each subsequent test, the visual presentation terminal sequentially displays two gratings, with the first grating oriented as follows: The orientation of the second grating is or Each grating lasts for 200 ms, with a 600 ms interval between two gratings. The post-test does not output associated or unassociated sounds to avoid reactivating the sound cues during the visual measurement phase. The user determines whether the second grating rotates clockwise or counterclockwise relative to the first grating, and updates the posterior distribution of the QUEST adaptive ladder based on the correct or incorrect result. The visual discrimination threshold after reactivation can be obtained using the same method as in step S3 to obtain the corresponding pre-test threshold.

[0127] S62. Calculate the improvement in visual performance based on the visual discrimination threshold before reactivation and the visual discrimination threshold after reactivation. The improvement in visual performance is calculated according to the following formula:

[0128] ;

[0129] in, Indicates condition The visual discrimination threshold before reactivation Indicates condition The visual discrimination threshold after reactivation To improve visual representation, indicate conditions. The percentage increase in visual performance.

[0130] when When the value is positive, it means that the post-test threshold is lower than the pre-test threshold, which means that the orientation difference required for the user to achieve the same 75% accuracy is reduced, and the visual discrimination performance is improved.

[0131] In this embodiment, as Figure 7 The diagram shows the measurement of visual discrimination threshold and evaluation of visual feature specificity before and after reactivation, under the following conditions. The following three conditions can be included: 1. Same Orientation, Same Location (SOSL): Using the target orientation and target location, this condition is used to evaluate the improvement in the target's visual features themselves. 2. Same Orientation, Different Location (SODL): Using the target orientation, but presenting the visual stimulus in a location different from the target location, this condition is used to evaluate location migration. 3. Different Orientation, Same Location (DOSL): Using the target location, but employing an orientation that differs from the target orientation by a preset angle. In this embodiment, an orientation that differs from the target orientation by 90° can be used to evaluate orientation migration. The visual discrimination threshold before reactivation, the visual discrimination threshold after reactivation, and the corresponding improvement in visual performance can be calculated for the three conditions: SOSL, SODL, and DOSL. When the improvement in visual performance under the SOSL condition is positive, and the SODL and DOSL conditions do not show the same magnitude of improvement, the result is marked as a visual perception enhancement specific to the target orientation and target location. When the SODL condition also shows an improvement, location migration is recorded. When the DOSL condition also shows an improvement, orientation migration is recorded. The results of migration conditions and the improvement under the target conditions are saved separately and are not combined into the same visual enhancement index.

[0132] Steps S1 to S5 are used to establish and verify cross-modal mapping, measure the visual discrimination threshold before reactivation, select the reactivation path, and implement reactivation. They are not used to determine whether visual perception is enhanced. Step S6 evaluates whether visual perception is enhanced and its specificity by comparing the visual discrimination threshold before and after reactivation and calculating the improvement in visual performance.

[0133] S7. Calculate the effective number of reactivation trials and the visual enhancement efficiency per effective trial. If the improvement in visual performance or the visual enhancement efficiency per effective trial does not meet the preset conditions, return to step S5 to update and optimize the reactivation strategy.

[0134] S71. Set a valid flag for each reactivation attempt and count the number of valid reactivation attempts. The valid flag can be represented as:

[0135] ;

[0136] in, Indicates the first Each trial checks whether a mapping with a valid status of 1 and the latest version number is used. This indicates whether the sound was output completely. This indicates whether the user's status meets the requirements of the selected path. This indicates whether an abnormal pause was not triggered during the trial. This indicates whether the quality of physiological signals or state monitoring signals has reached a threshold. This indicates whether the trial has a complete timestamp and event record. All the above parameters are binary parameters, taking the value 1 if the condition is met and 0 if the condition is not met. Only when all conditions are met... In the sleep path, the user's status meets the requirements if they are in a stable N3 sleep state when the sound begins, and there are no abnormalities in wakefulness, body movement, or sleep stage during the trial. In the wakefulness path, the user's status meets the requirements if the wakefulness working mode is effective, and the sound intensity is higher than the user's perceptible threshold but does not exceed the safe output limit.

[0137] Specifically, the number of effective reactivation attempts Calculate according to the following formula:

[0138] ;

[0139] in, This indicates the actual number of reactivation attempts performed. This represents the number of valid reactivation attempts that pass the checks for mapping validity, status, output integrity, pause events, signal quality, and record integrity. Trials excluded due to awakening, phase transition, physical movement, mapping failure, interrupted audio output, abnormal signal quality, or missing data are not counted. .

[0140] S72. Calculate the visual enhancement efficiency per unit of effective reactivation trials based on the number of effective reactivation trials and the percentage improvement in visual performance, for any user and any test conditions. unit effective trial visual enhancement efficiency Calculate according to the following formula:

[0141] ;

[0142] in, Indicates condition The percentage improvement in visual performance This indicates the number of valid reactivation attempts for the user. This indicates the percentage improvement in visual performance corresponding to each effective reactivation trial.

[0143] Step S71 counts the number of valid reactivation attempts N that pass the checks for mapping validity, user status, complete audio output, abnormal pause, signal quality, and recording integrity. eff Step S72 further calculates the effective visual enhancement efficiency (ETN) per unit of trial, enabling normalized comparisons of sleep reactivation, wakefulness reactivation, and wakefulness visual training under different actual number of trials completed.

[0144] S73. Update the reactivation strategy. If the visual performance improvement (VPI) and the visual enhancement efficiency per unit effective trial do not meet the preset conditions, return to step S5 and adjust the subsequent reactivation path, associated cues, output dose, and state gating parameters. When the visual performance improvement (VPI) reaches the preset statistical criterion and the visual enhancement efficiency per unit effective trial (ETN) increases by a preset amount compared to the original ETN, the current reactivation parameters are considered to simultaneously meet the requirements for visual enhancement effect and reactivation efficiency. Then, stop optimizing the reactivation process. Through these adjustments, the subsequent reactivation process is adaptively optimized based on the obtained visual enhancement effect and unit trial efficiency.

[0145] S731 If the visual performance improvement amount and the visual enhancement efficiency per unit of effective trial do not meet the preset conditions, return to step S5 to adjust the subsequent reactivation path, associated cues, output dose and state gating parameters.

[0146] The visual perception enhancement is evaluated by the visual performance improvement (VPI) and the extent of the improvement. Step S7 then counts the number of effective trials verified by mapping effectiveness, brain state, pause events, and signal quality, calculates the visual enhancement efficiency per trial, and updates the sleep or wakefulness work path, the selection of associated cues, output dose, or state gating parameters based on the visual performance improvement and the visual enhancement efficiency per trial.

[0147] Adjust the reactivation path selection. When the user allows the sleep path, can achieve a stable N3 sleep state, and the visual enhancement efficiency per unit of effective trials of the historical sleep path is higher than that of the wake path by a preset difference, the sleep path is selected first. When the user cannot enter a stable N3 sleep state, the signal quality is insufficient, or immediate reactivation is required, the wake path is selected.

[0148] Adjust the selection of audio cues to only select records with a valid status of 1, the latest version number, and a correlation quality that meets the criteria, based on the visual features of the target. Consistent associated clues. Reactivation will not be initiated if no valid mapping exists.

[0149] Adjusting the output dose allows for the determination of candidate output doses for the next stage based on historical visual performance enhancement, historical effective reactivation trial counts, and historical visual enhancement efficiency per effective trial.

[0150] In one implementation, the sleep pathway is implemented by forming a short sound sequence of 10 trials. The actual number of trials completed is still counted as the number of effective reactivation trials. The above candidate test count is for initial dose estimation only. Actual output is still subject to user status, sound safety limits, abnormal pause conditions, and preset minimum and maximum dose limitations.

[0151] Adjusting the state gating parameters, the sleep path only occurs when the stable N3 duration reaches [a certain value]. After activation. If awakening, stage abnormalities, or signal quality degradation occur, reduce sound intensity, extend the trial interval, or pause output. During the awakening path, maintain sound intensity above the user's perceptible sound threshold but not exceeding the safe output limit.

[0152] S732. Update the visual enhancement efficiency per unit of effective trials until the visual performance improvement (VPI) reaches the preset statistical criterion and the visual enhancement efficiency per unit of effective trials improves by a preset margin compared to the original visual enhancement efficiency per unit of effective trials. Then, stop the optimization and reactivation process. Historical visual enhancement efficiency per unit of trials can be updated using the following formula:

[0153] ;

[0154] in, This indicates the updated historical estimate. This indicates the historical estimate before the update. This indicates the efficiency of visual enhancement per unit of effective trials obtained in the current task. This represents the update weight between 0 and 1.

[0155] When the visual performance improvement (VPI) is greater than 0, and its statistical result relative to 0 meets the preset statistical criteria (e.g., a p-value less than 0.05), the visual enhancement effect is considered to have met the preset enhancement criteria. Based on this, when the visual enhancement efficiency (ETN) per unit of effective trials increases by a preset margin relative to the original ETN, the current reactivation parameters are considered to simultaneously meet the requirements for visual enhancement effect and reactivation efficiency. Therefore, further optimization is stopped, and the current reactivation parameters are maintained.

[0156] like Figure 8The diagram illustrates a comparison of the visual enhancement efficiency per unit of effective trials between sleep reactivation and dose-matched awake visual training. In this embodiment, each group included 20 users in the comparison between sleep reactivation and dose-matched awake visual training. The sleep reactivation group completed an average of 60.2 ± 7.04 associated sound trials, while the dose-matched awake visual training group completed an average of 58.0 ± 5.41 visual discrimination training trials; no significant difference was observed between the two groups in the number of trials completed. During sleep reactivation, no target visual stimulus was presented, and users were not required to perform key presses. During awake visual training, users were required to continuously fixate on the visual stimulus and complete visual discrimination responses.

[0157] The improvement in SOSL visual performance was 13.75±1.42% in the sleep reactivation group and 3.08±3.07% in the dose-matched wakeful visual training group. The intergroup comparison results are as follows:

[0158]

[0159] Visual enhancement efficiency per unit of effective trials is calculated by dividing the improvement in visual performance for each user by the actual number of effective trials for that user. (Sleep reactivation group) The dose-matched group had a dose of 0.332 ± 0.074% per trial, while the dose-matched awake visual training group had a dose of 0.087 ± 0.066% per trial. The intergroup comparison results are as follows: ;

[0160] Two-way ANOVA showed that the main effect of the operating method was: ;

[0161] The main effects of paired experiments are: ;

[0162] The interaction is as follows: ;

[0163] The above results indicate that, at the exposure level of this embodiment, sleep reactivation can achieve a high visual performance enhancement and enhancement efficiency per effective trial without relying on active operation and visual channel occupancy.

[0164] like Figure 9 The diagram shows a comparison of visual enhancement efficiency per unit of effective trials between sleep reactivation and wakefulness reactivation. In this embodiment, after normalization based on the number of effective trials, the sleep reactivation group... The reactivation rate was 0.332 ± 0.074% per trial in the control group and 0.061 ± 0.009% per trial in the awake reactivation group. The intergroup comparison results are as follows: .

[0165] The wake-up reactivation implementation includes 240 sound trials and requires the user to make a choice between raising or lowering the pitch.

[0166] Sleep reactivation was performed in a stable N3 sleep state using short sequences of 10 trials, averaging approximately 60 effective trials, and did not require user task response. Therefore, within the parameter range and exposure level of this embodiment, sleep reactivation achieved a similar overall visual performance improvement to wakefulness reactivation with fewer effective trials, and exhibited higher efficiency per effective trial. Wakefulness reactivation and wakefulness visual training comprising 240 trials showed no significant difference in efficiency per trial. The percentages were approximately 0.061 ± 0.009% per trial. The comparison results are as follows: When users are unable to perform sleep monitoring, have not entered a stable N3 sleep state, or need to complete training immediately, wake-up reactivation can still be an optional path.

[0167] In the sleep reactivation embodiment of the present invention, associated cues are output during stable N3 sleep. The reactivation phase does not present a target grating, nor does it require the user to perform visual discrimination, auditory judgment, or key press responses. Under these conditions, the improvement in visual performance in SOSL was 13.75 ± 1.42%; in dose-matched wakeful visual training conditions, the user needs to continuously view visual stimuli and complete visual discrimination responses, resulting in an improvement of 3.08 ± 3.07%. The difference between the two groups was t(38) = 3.152, P = 0.003, d = 0.997. This demonstrates that under the conditions tested in this embodiment, sleep reactivation can still achieve improved visual performance without continuously occupying the visual channel and without requiring active task responses, and its effect is significantly better than visual training.

[0168] Within the sleep condition, the improvement in visual performance corresponding to associated cues was 13.75±1.42%, while the change for unassociated cues was -12.16±2.71%, showing a significant difference. In another intra-user comparison, the improvement in visual performance corresponding to associated cues was 10.87±3.03%, while the change for unassociated cues was -14.59±4.22%, also showing a significant difference. These results indicate that not all auditory cues produce changes in visual performance in the same direction. Therefore, step S2 sets up association quality verification, and step S5 excludes unassociated cues by default in the enhancement sequence to reduce the risk of incorrect mapping or unverified sounds entering the reactivation phase. In the comparison of imperceptible sounds in the waking state, no reliable improvement in visual performance was observed for associated cues, unassociated cues, and no sound conditions, with a conditional main effect of F(2,57)=2.665, P=0.078; no stable enhancement was observed after increasing the presentation dose of imperceptible sounds. Therefore, step S51 sets a sound perception threshold gating so that wake-up reactivation is only implemented when the sound intensity reaches a level that is perceptible to the user and does not exceed the safe output limit.

[0169] This invention can be used for non-therapeutic visual ability training, state-dependent human-computer interaction, and visual performance evaluation. Equivalent substitutions for stimulus types, state classifiers, or adaptive ladder algorithms, without departing from the core pathways of visual feature-audio cue verification mapping, targetless visual input reactivation, brain state gating, and visual threshold evaluation, are all embodiments of this invention.

Claims

1. A visual perception enhancement method based on cross-modal cue reactivation, characterized in that, Including the following steps: S1. Under the condition that the target visual stimulus and candidate sound cues are presented simultaneously, perform association learning to establish the cross-modal correspondence between the target visual features and the candidate sound cues, and form a candidate cross-modal mapping record to be verified. S2. Perform a cross-modal association verification task. Calculate the association quality of candidate cross-modal mapping records based on the user's response to the cross-modal association verification task. When the association quality reaches a preset value, store the candidate cross-modal mapping records as verified cross-modal mappings and mark the candidate audio cues as associated cues. S3. Perform a silent adaptive visual discrimination task to obtain the visual discrimination threshold of the target visual features before reactivation. S4. Collect the user's EEG and EEG signals, determine whether the user is in a stable N3 sleep state or in a waking state based on the user's EEG and EEG signals, and select a reactivation path according to the user's state. S5, without presenting the target visual stimulus, uses pre-associated cues that are linked to the target visual features to reactivate the corresponding visual representations in the validated cross-modal mapping. S6. After the reactivation phase ends, calculate the visual discrimination threshold after reactivation, and calculate the improvement in visual performance based on the visual discrimination threshold before reactivation and the visual discrimination threshold after reactivation. S7. Calculate the number of effective reactivation trials and the visual enhancement efficiency per effective trial. If the improvement in visual performance or the visual enhancement efficiency per effective trial does not meet the preset conditions, return to step S5 to update and optimize the reactivation strategy.

2. The visual perception enhancement method based on cross-modal cue reactivation according to claim 1, characterized in that, Step S1 includes: S11. Define the target visual features that need to be enhanced, and define the candidate audio cue parameters; S12. Control the visual presentation terminal to present visual stimuli with target visual features, and control the sound cue output unit to output candidate sound cues synchronized with the visual stimuli, perform visual orientation discrimination task and respond through the response input unit, and establish cross-modal correspondence between target visual features and candidate sound cues. S13. Based on the cross-modal correspondence between the target visual features and candidate acoustic cues, establish a candidate cross-modal mapping record, which is represented as follows: ； in, This represents a cross-modal mapping record. Identify the visual features of the target. The audio parameters for candidate audio cues. For related quality, To map version numbers, To map the timestamps that have been verified and written to the database, This is a valid mapping state.

3. The visual perception enhancement method based on cross-modal cue reactivation according to claim 2, characterized in that, Step S2 includes: S21. Perform cross-modal association verification task, control the sound cue output unit to output sound stimuli composed of learned reference tones, control the visual presentation terminal to present the target grating and interference grating in the left and right visual fields respectively, and the user determines whether the target grating is located in the left or right visual field and responds through the response input unit. S22. Calculate the number of correct responses based on the user's response to the cross-modal association verification task, and calculate the association quality based on the number of correct responses.

4. The visual perception enhancement method based on cross-modal cue reactivation according to claim 3, characterized in that, The associated quality is represented as follows: ； in, This represents the lower bound of the 95% confidence interval for matching accuracy. Indicates the matching accuracy. Indicates the number of correct responses. Indicates the number of valid verification tests. This represents the critical value corresponding to the confidence level in the standard normal distribution.

5. The visual perception enhancement method based on cross-modal cue reactivation according to claim 1, characterized in that, Step S3 includes: S31. Establish an initial threshold probability distribution, and use the QUEST adaptive ladder method to control the visual presentation terminal to sequentially present two gratings, recording the user's position on the [missing information]. The response results of each trial; the orientation difference is selected as the visual discrimination threshold based on the current posterior distribution; S32. Update the posterior distribution of the threshold, and determine the occurrence of the first [condition] under the candidate threshold and orientation difference based on the psychometric function. The likelihood of the response results for each trial is then updated according to the Bayesian rule to determine the threshold posterior distribution. S32. After completing the preset number of trials, estimate the visual discrimination threshold of a single QUEST adaptive ladder based on the final posterior distribution.

6. The visual perception enhancement method based on cross-modal cue reactivation according to claim 1, characterized in that, Step S4 includes: S41. The user's brainwave and electrooculogram (EOG) signals are continuously acquired through the EEG and EOG acquisition unit, and the brainwave and EOG signals are divided according to a preset time window. S42. Based on the user's EEG and EOG signals, determine whether the user has entered a stable N3 sleep state and whether the user is awake. S43. When the user has entered a stable N3 sleep state, and the EEG and EEG signals are continuous and recorded completely, without any obvious artifacts or signal interruptions that affect the determination of sleep state, select the sleep reactivation path; when the user is awake, sleep monitoring conditions are not available, or reactivation needs to be completed immediately, select the awake reactivation path.

7. The visual perception enhancement method based on cross-modal cue reactivation according to claim 6, characterized in that, Step S5 includes: S51. When selecting the wake-up reactivation path, complete the sound discrimination task and output multiple wake-up reactivation sound trials. Each wake-up reactivation sound trial outputs two sound stimuli in sequence, without presenting visual stimuli containing the target orientation and target location, and determine the sound perception threshold and safe output upper limit. S52. When selecting a sleep reactivation path, based on the associated cues in the verified cross-modal mapping, set multiple sound reactivation trials into a preset short sleep sound sequence and output the short sleep sound sequence; the user is not required to complete visual discrimination, auditory judgment or key response.

8. The visual perception enhancement method based on cross-modal cue reactivation according to claim 1, characterized in that, Step S6 includes: S61. Perform a silent visual post-test task and calculate the visual discrimination threshold after reactivation. S62. Calculate the improvement in visual performance based on the visual discrimination threshold before reactivation and the visual discrimination threshold after reactivation. The improvement in visual performance is calculated according to the following formula: ； in, Indicates condition The visual discrimination threshold before reactivation Indicates condition The visual discrimination threshold after reactivation To increase the quantity of visual presentation.

9. The visual perception enhancement method based on cross-modal cue reactivation according to claim 1, characterized in that, Step S7 includes: S71. Set a valid flag for each reactivation attempt and count the number of valid reactivation attempts; S72. Calculate the visual enhancement efficiency per unit of effective reactivation trials based on the effective number of reactivation trials and the percentage improvement in visual performance. S73. If the visual performance improvement amount and the visual enhancement efficiency per unit effective trial do not meet the preset conditions, return to step S5 and adjust the reactivation path, associated cues, output dose, and state gating parameters. When the visual performance improvement amount (VPI) reaches the preset statistical criterion and the visual enhancement efficiency per unit effective trial increases by a preset amount compared to the visual enhancement efficiency per unit effective trial before adjustment, it is considered that the current reactivation parameters simultaneously meet the requirements of visual enhancement effect and reactivation efficiency, and the strategy for optimizing the reactivation process is stopped.

Images