Vibration image sleep cycle dynamic analysis method based on transformer time series modeling

Abstract

The application discloses a vibration image sleep cycle dynamic analysis method based on a Transformer timing modeling, and comprises the following steps: collecting a video sequence, performing detection and noise evaluation, and obtaining a corrected video sequence; performing target region detection on the corrected video sequence to generate a vibration amplitude time sequence; slicing the vibration amplitude time sequence to construct an energy window sequence; constructing an improved Transformer combined encoder; performing convolution coding on the energy window sequence in a stable pole state space convolution unit and outputting a convolution response sequence; inputting the energy window sequence into an energy conservation attention unit to form a context feature sequence; performing parallel fusion on the convolution response sequence and the context feature sequence to output a timing implicit representation sequence; calculating sleep stage labels and confidence of the timing implicit representation sequence to generate a sleep cycle analysis report. The application realizes vibration image sleep cycle dynamic analysis.

Description

Technical Field

[0001] This invention relates to the field of signal analysis technology, and in particular to a method for dynamic analysis of vibration image sleep cycles based on Transformer time-series modeling. Background Technology

[0002] Current sleep cycle assessments primarily rely on polysomnography (PSG) and wearable accelerometers. PSG, using multi-channel signals such as EEG, eye movement, and electromyography (EMG), performs manual or semi-automatic staging with high accuracy, but it is complex to deploy, causes significant interference to subjects, and suffers from low cost and poor adaptability to various scenarios. Wearable accelerometers, while convenient to deploy, are sensitive to body movements and insufficiently reflect subtle respiratory and facial vibrations, easily leading to stage boundary drift and misjudgments. Recent video-based non-contact methods extract micro-vibrations using inter-frame difference or optical flow, avoiding electrode and wear constraints. However, over extremely long periods overnight, these methods are susceptible to fluctuations in illumination, noise accumulation, and camera auto-gain, resulting in unstable temporal features and distorted energy statistics. Furthermore, conventional convolutional or recurrent networks lack sufficient constraints on long-term dependencies and cross-window consistency.

[0003] With the development of temporal modeling, Transformer has been used for sleep staging and physiological signal analysis. It relies on global attention to capture long-distance dependencies and improves the sensitivity to stage transitions. However, standard attention mechanisms tend to produce non-physical cross-window aggregation at strong noise or local peaks, lack constraints on short-window energy budgets, and are prone to introducing spurious energy amplification and boundary misjudgment.

[0004] On the other hand, state space or one-dimensional convolution can provide stable filtering and multi-timescale memory in a local range, but a single path is difficult to simultaneously take into account global dependence and physical consistency, and there are problems of numerical drift and difficulty in choosing the memory scale over the entire night sequence.

[0005] Therefore, how to provide a method for dynamic analysis of vibration image sleep cycles based on Transformer time-series modeling is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a dynamic analysis method for vibration image sleep cycles based on Transformer temporal modeling. This invention completes dynamic analysis of the entire night's sleep cycle, enhances long-term dependence and numerical stability through stable pole convolution, suppresses cross-window mis-aggregation and spurious energy through energy conservation attention, improves the accuracy of staging, the reliability and consistency of boundary determination, and outputs cycle curves and confidence scores. It is interpretable, has low deployment cost, and is more robust to light and noise fluctuations.

[0007] The method for dynamic analysis of vibration image sleep cycles based on Transformer time-series modeling according to embodiments of the present invention includes the following steps: Under conditions of fixed frame rate, fixed resolution and constant illumination, a video sequence of the test object was acquired overnight. Brightness stability was detected and noise was evaluated for each frame. Frames that did not meet the quality threshold were removed and timestamps were recorded to obtain a corrected video sequence. Target region detection and cross-frame tracking are performed on the corrected video sequence to locate the region of interest of the head and face. The difference between adjacent frames is calculated and time accumulation and normalization are performed to generate a vibration amplitude time series that reflects the micro-vibration changes. The vibration amplitude time series is sliced ​​according to a fixed window length and a fixed step size, and the energy value and frequency characteristics of each time window are calculated to construct an energy window sequence. An improved Transformer joint encoder is constructed, which includes two substructures: a stable pole state space convolutional unit and an energy-conserving attention unit. The former is used to extract long-term dependency features and ensure the numerical stability of temporal modeling, while the latter is used to capture global dependencies and constrain the energy conservation of attention. The two are connected in parallel to form a joint encoding structure. A multi-scale pole spectrum array is established in the state space convolution unit of the stable pole. The pole groups with different time constants are selectively activated through a sparse gating mechanism. The energy window sequence is convolved and encoded to output the convolution response sequence. The energy window sequence is input into the energy-conserving attention unit. In the energy-conserving attention unit, an attention mask is constructed for each window based on the short window energy budget. The attention weights in the same window and adjacent windows are masked and recalibrated to form a context feature sequence. The convolutional response sequence and the context feature sequence are fused in parallel, and forward energy prior transfer, backward inhibition feedback and consistency correction are performed to output the temporal hidden representation sequence. Calculate the sleep stage labels and confidence levels corresponding to each time slice of the time-series implicit representation sequence, and generate a sleep cycle analysis report.

[0008] Optionally, obtaining the corrected video sequence specifically includes: A fixed-position camera was set up in the natural sleep environment of the subject. The light source was adjusted to make the screen brightness uniform. The preview was turned on and the shooting parameters, including the frame rate, resolution, exposure and white balance, were locked. The time recording and calibration functions were enabled. Short static images were captured for reference. After eliminating reflections and obstructions, the nightly recording was started to generate the original video frame sequence and timestamp. During the acquisition process, quality inspection is performed on each frame of the original video frame sequence, the overall brightness change rate and the image noise level are calculated, and frames with brightness fluctuations or noise exceeding the set threshold are judged as unqualified and removed. At the same time, the detection results and time information are recorded to form a set of qualified frames. The qualified frames are arranged in chronological order and the time intervals are checked. Adjacent intervals are compared with the theoretical frame intervals. If the interval exceeds the tolerance, it is only registered as an interval abnormality and no composite frame is inserted. The sorted qualified frames are packaged to obtain the corrected video sequence.

[0009] Optionally, the process of generating the vibration amplitude time series specifically includes: Read the corrected video, locate the region of interest of the head and face in the first frame, divide the left and right sub-regions according to the midline of the bridge of the nose, start cross-frame tracking and update the region position and pixel mask frame by frame, calculate the overlap ratio with the previous frame, trigger fast relocalization if it is lower than the threshold, and output the region coordinates and mask sequence sorted by time. Within a defined region, the absolute value of the grayscale difference is calculated pixel by pixel in adjacent frames. Time accumulation is performed using a sliding method with a fixed window length and a fixed step size, and normalization is performed using the start and end duration of the window. The vibration amplitude per unit time for the whole face, left side, and right side is obtained respectively. At the same time, the center time of each window is recorded as a time label. The time stamps and the corresponding vibration amplitude values ​​of the full face, left side, and right side are arranged in chronological order to form a vibration amplitude time series. Each window is marked with a region overlap ratio and tracking status. Low-overlap or mismatched windows are marked with low quality, and the processing logs and parameter configurations are saved.

[0010] Optionally, the construction of the energy window sequence specifically includes: Based on the vibration amplitude time series and time label, the vibration amplitude time series includes three vibration amplitudes per unit time for the whole face, left side and right side. A fixed window length and a fixed step size are set, and a continuous window is generated by sliding according to time. The three amplitudes and time points in each window are extracted, and the center time of the window is recorded as a time marker. For each window, calculate the energy values ​​of the whole face, left side, and right side separately. Square the vibration amplitude of each sampling point in the window, sum them up by adjacent time intervals, and divide by the window duration. Obtain the discrete spectrum energy distribution based on the whole face amplitude sequence in the window. Calculate the main frequency, spectrum centroid, and second-order bandwidth in sequence, and calculate the energy ratio of low frequency and high frequency. At the same time, obtain the calculation results of the left and right amplitude sequences. Based on the calculation results of the left and right paths, the energy asymmetry and frequency asymmetry are calculated, and a quality mark is attached to the window to mask low-quality segments. The time stamp, the three energy values, the main frequency and spectral centroid of the whole face, the second-order bandwidth, the proportion of low-frequency and high-frequency energy, and the two types of asymmetry are assembled into energy window items and summarized in chronological order to form an energy window sequence.

[0011] Optionally, the construction of the improved Transformer joint encoder specifically includes: The input layer and sequence organization of the improved Transformer joint encoder are constructed. For each time window of the energy window sequence, energy window items are extracted and assembled into feature vectors in a unified order. Position encoding vectors are generated for time identifiers and added to the feature vectors to form input entries. All entries are arranged into an input sequence according to time. At the same time, a corresponding index table and field mapping table are established to keep the input dimension consistent within the encoder. Set up a stable pole state space convolutional unit, which includes a pole parameter library, a sparse gate layer, a convolution kernel generator, and a convolution channel. By establishing a hidden state buffer and a linear readout layer, the absolute value of the eigenvalues ​​of the state update matrix is ​​defined to be less than one to ensure pole stability. Multiple sets of pole parameters are preset to cover different time constant ranges. The sparse gate layer is configured to selectively activate pole groups according to time position, generate corresponding convolution kernels, and process the input sequence in a time-invariant convolution manner inside the unit. An energy-conserving attention unit is set up, including a projection layer, a similarity channel, a mask generator, a budget register, a recalibrator, and a weighted convergence channel. Linear projection layers for query, key, and value are configured to generate vector features. A similarity calculation channel and an energy budget register are established. An attention mask is constructed to limit the aggregation range of windows in the same window and adjacent windows, while other positions are masked. A weighted recalibrator is established to normalize and scale the similarity results according to the energy budget. A weighted convergence channel is configured to linearly combine the value vectors according to the weights, ensuring consistency with the input sequence in terms of time index and feature dimension.

[0012] Optionally, the output of the convolutional response sequence specifically includes: The encoder input sequence arranged in time is read from the input layer of the improved Transformer co-encoder. The input entries are obtained by adding the position information of the time marker and the energy window feature vector component by component. In the state space convolution unit of the stable pole, the hidden state buffer is initialized for each group of poles so that the absolute value of all eigenvalues ​​of the state update matrix is ​​less than one. The current input entries are sent to each pole group in sequence according to the time index and accumulated to the hidden state. The convolution response of the current pole group at the current time position is obtained by linear reading. At the same time, the convolution kernel generator generates an equivalent one-dimensional time-invariant convolution kernel according to the pole parameters and the preset kernel length. The convolution response of all pole groups at each time step is temporarily stored as a temporary sequence of convolution response of each pole group, and the corresponding time index is retained to align with subsequent gating and convergence. Based on the temporary sequence of convolutional responses of each pole group and the input entries at the same time position, the gating score of each time position for all pole groups is calculated in the sparse gating layer. The initial gating weights are obtained by exponential normalization. Only a preset number of pole groups are retained as the activation set from high to low. The initial weights of the selected pole groups are retained as effective weights. The weights of the non-selected pole groups are reset to zero. The sum of the effective weights at the same time position is constrained to not exceed one. The sparse gating weight sequence is output. At each time point, read the temporary sequence of convolutional responses of each pole group and the sparse gating weight sequence. Perform linear weighted summation on the convolutional responses of all activated pole groups at the time point according to the effective weights to obtain the output value of the state space convolutional branch of the stable pole. Arrange the output values ​​of all time points in chronological order to form a convolutional response sequence.

[0013] Optionally, the output process of the context feature sequence specifically includes: Read the energy window sequence, calculate the short window energy budget for each time position, normalize it to the interval of zero to one, build an aggregateable index table containing only the index of the current window and adjacent windows, and send the input entries, energy budget and aggregateable index table into the energy conservation attention unit. Within the energy-conserving attention unit, query vector, key vector, and value vector are generated sequentially for each input entry at each time position. All three are obtained from their respective linear projection layers and have the same dimension. The unnormalized similarity score of the current time position to all candidate positions is calculated, and a binary mask is generated based on the aggregateable index table. Scores that do not belong to the current window and adjacent windows are marked as invalid. Row-wise exponential normalization is performed on the remaining valid scores to obtain the initial attention weights. The sum of the weights in the current row is calculated and compared with the corresponding energy budget to obtain a scaling factor not exceeding one. The initial attention weights are scaled item by item using the scaling factor to form a recalibrated weight distribution that satisfies the upper limit of the budget. Traverse the time position, multiply the recalibration weights with the corresponding value vectors one by one and sum them to obtain the context feature vector of the current position. Then, assemble the context feature vectors of all time positions into a context feature sequence in chronological order.

[0014] Optionally, the output of the temporal implicit representation sequence specifically includes: Based on the convolutional response sequence and the context feature sequence, the corresponding energy budget and recalibration coefficients are obtained simultaneously. After being aligned by time index, they are fused. The two vectors at the same time position are combined proportionally and weighted. The weights are automatically allocated according to the energy ratio of the two signals, with the higher energy vector having a slightly higher weight and the lower energy vector having a slightly lower weight, in order to obtain the initial fusion result while maintaining the time order. Energy adjustment and suppression control are applied to the initial fusion result. The amplitude of the fusion signal is scaled according to the energy budget value at each time position to keep the total energy within the budget range. The energy of the convolution branch and the attention branch at the same position are compared. When the energy of the attention branch is too high, its proportion is automatically reduced to prevent over-aggregation. At the same time, the output of the convolution branch is kept stable, and finally, the fusion sequence after energy balance is generated. Consistency correction is performed on the fused sequence. The deviation between the energy estimate and the corresponding budget at the current time point is detected. It is determined whether the deviation exceeds the allowable range. If it does, the amplitude of the current output is finely adjusted in the direction of the deviation. If it does not exceed the allowable range, it remains unchanged. The correction process is performed only once. The corrected results are output sequentially in time order to form a temporal implicit representation sequence.

[0015] Optionally, the generation of the sleep cycle analysis report specifically includes: Read the temporal implicit representation sequence, analyze the feature changes of each time slice in chronological order, determine the sleep stage type corresponding to the current time slice, and record the time index and duration of the current stage. At the same time, calculate the matching confidence when each stage occurs to reflect the reliability of stage identification. Adjacent time slices belonging to the same stage are continuously merged to form stage segments. The start time, end time and duration of each segment are counted. The proportion of each stage in the total duration of sleep throughout the night is calculated. The transition relationship and number of transitions between stages are recorded. The distribution of major stages, including wakefulness, light sleep, deep sleep and REM sleep, is identified. Based on the distribution of major stages, a periodic curve of sleep stages changing over time is plotted with time as the horizontal axis. A band of confidence changes for each stage is superimposed to show stage stability and transition clarity. The proportion of each stage, average duration, and number of stage transitions are summarized. Combined with energy window features and time index, a whole-night sleep cycle analysis report is generated.

[0016] The beneficial effects of this invention are: This invention takes a corrected video sequence as input, combines a vibration amplitude time series and an energy window sequence, and uses an improved Transformer joint encoder to achieve stable modeling of long-term time series throughout the night. The stable pole state space convolution unit ensures numerical stability and multi-timescale memory, while the energy conservation attention unit suppresses cross-window mis-aggregation and pseudo-energy amplification under the constraints of short-window energy budget and same-window and adjacent window masks, making the boundary determination of the sleep stage more robust. After parallel fusion of the convolutional response sequence and the context feature sequence, the present invention significantly reduces the drift and jump at the stage switching point through forward energy prior transfer, reverse suppression feedback and first consistency correction, improves the continuity and credibility of the temporal implicit representation sequence, and thus improves the accuracy of night-time stage segmentation and retest consistency. This invention maintains the alignment of input and output names with time indices throughout the entire process, and includes traceable indicators such as quality markers and left-right asymmetry, which facilitates the masking and interpretation of abnormal segments and enhances clinical interpretability and engineering reproducibility. This invention uses non-contact video acquisition and algorithm-end constraints to replace the complex arrangement of multi-channel or wearable devices, reducing deployment and usage costs, and is more robust to light fluctuations and noise, making it easier to use continuously for a long time in scenarios such as home and hospital wards. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the vibration image sleep cycle dynamic analysis method based on Transformer temporal modeling proposed in this invention; Figure 2 This is a schematic diagram of the algorithm structure of the vibration image sleep cycle dynamic analysis method based on Transformer time-series modeling proposed in this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0019] refer to Figure 1-2 A method for dynamic analysis of vibration image sleep cycles based on Transformer time-series modeling includes the following steps: Under conditions of fixed frame rate, fixed resolution and constant illumination, a video sequence of the test object was acquired overnight. Brightness stability was detected and noise was evaluated for each frame. Frames that did not meet the quality threshold were removed and timestamps were recorded to obtain a corrected video sequence. Target region detection and cross-frame tracking are performed on the corrected video sequence to locate the region of interest of the head and face. The difference between adjacent frames is calculated and time accumulation and normalization are performed to generate a vibration amplitude time series that reflects the micro-vibration changes. The vibration amplitude time series is sliced ​​according to a fixed window length and a fixed step size, and the energy value and frequency characteristics of each time window are calculated to construct an energy window sequence. An improved Transformer joint encoder is constructed, which includes two substructures: a stable pole state space convolutional unit and an energy-conserving attention unit. The former is used to extract long-term dependency features and ensure the numerical stability of temporal modeling, while the latter is used to capture global dependencies and constrain the energy conservation of attention. The two are connected in parallel to form a joint encoding structure. A multi-scale pole spectrum array is established in the state space convolution unit of the stable pole. The pole groups with different time constants are selectively activated through a sparse gating mechanism. The energy window sequence is convolved and encoded to output the convolution response sequence. The energy window sequence is input into the energy-conserving attention unit. In the energy-conserving attention unit, an attention mask is constructed for each window based on the short window energy budget. The attention weights in the same window and adjacent windows are masked and recalibrated to form a context feature sequence. The convolutional response sequence and the context feature sequence are fused in parallel, and forward energy prior transfer, backward inhibition feedback and consistency correction are performed to output the temporal hidden representation sequence. Calculate the sleep stage labels and confidence levels corresponding to each time slice of the time-series implicit representation sequence, and generate a sleep cycle analysis report.

[0020] This invention uses the structured representation of energy window sequences as a link, integrates multi-timescale memory of stable pole convolution with budget constraints of energy conservation attention, suppresses spurious vibrations caused by light and noise, reduces cross-window mis-aggregation and stage drift, significantly improves the accuracy, boundary stability and test-retest consistency of overnight sleep staging, and enhances interpretability and clinical reusability with confidence-labeled periodic reports.

[0021] In this embodiment, obtaining the corrected video sequence specifically includes: A fixed-position camera was set up in the natural sleep environment of the subject. The light source was adjusted to make the screen brightness uniform. The preview was turned on and the shooting parameters, including the frame rate, resolution, exposure and white balance, were locked. The time recording and calibration functions were enabled. Short static images were captured for reference. After eliminating reflections and obstructions, the nightly recording was started to generate the original video frame sequence and timestamp. During the acquisition process, quality inspection is performed on each frame of the original video frame sequence, the overall brightness change rate and the image noise level are calculated, and frames with brightness fluctuations or noise exceeding the set threshold are judged as unqualified and removed. At the same time, the detection results and time information are recorded to form a set of qualified frames. The qualified frames are arranged in chronological order and the time intervals are checked. Adjacent intervals are compared with the theoretical frame intervals. If the interval exceeds the tolerance, it is only registered as an interval abnormality and no composite frame is inserted. The sorted qualified frames are packaged to obtain the corrected video sequence.

[0022] This invention eliminates the accumulation of spurious vibration signals caused by illumination fluctuations and automatic gain control of equipment by performing brightness and noise quality checks at the acquisition end and generating corrected video sequences. This improves the stability and consistency of subsequent time series analysis from the source and reduces the propagation of sequence errors throughout the night.

[0023] In this embodiment, the process of generating the vibration amplitude time series specifically includes: Read the corrected video, locate the region of interest of the head and face in the first frame, divide the left and right sub-regions according to the midline of the bridge of the nose, start cross-frame tracking and update the region position and pixel mask frame by frame, calculate the overlap ratio with the previous frame, trigger fast relocalization if it is lower than the threshold, and output the region coordinates and mask sequence sorted by time. Within a defined region, the absolute value of the grayscale difference is calculated pixel by pixel in adjacent frames. Time accumulation is performed using a sliding method with a fixed window length and a fixed step size, and normalization is performed using the start and end duration of the window. The vibration amplitude per unit time for the whole face, left side, and right side is obtained respectively. At the same time, the center time of each window is recorded as a time label. The time stamps and the corresponding vibration amplitude values ​​of the full face, left side, and right side are arranged in chronological order to form a vibration amplitude time series. Each window is marked with a region overlap ratio and tracking status. Low-overlap or mismatched windows are marked with low quality, and the processing logs and parameter configurations are saved.

[0024] This invention performs robust cross-frame tracking within the region of interest on the face and accumulates the difference between adjacent frames to form a vibration amplitude time series, enabling minute physiological vibrations to be represented continuously in time and focused in space, significantly reducing the impact of background interference on the stage boundaries and transition judgment.

[0025] In this embodiment, the construction of the energy window sequence specifically includes: Based on the vibration amplitude time series and time label, the vibration amplitude time series includes three vibration amplitudes per unit time for the whole face, left side and right side. A fixed window length and a fixed step size are set, and a continuous window is generated by sliding according to time. The three amplitudes and time points in each window are extracted, and the center time of the window is recorded as a time marker. For each window, calculate the energy values ​​of the whole face, left side, and right side separately. Square the vibration amplitude of each sampling point in the window, sum them up by adjacent time intervals, and divide by the window duration. Obtain the discrete spectrum energy distribution based on the whole face amplitude sequence in the window. Calculate the main frequency, spectrum centroid, and second-order bandwidth in sequence, and calculate the energy ratio of low frequency and high frequency. At the same time, obtain the calculation results of the left and right amplitude sequences. Based on the calculation results of the left and right paths, the energy asymmetry and frequency asymmetry are calculated, and a quality mark is attached to the window to mask low-quality segments. The time stamp, the three energy values, the main frequency and spectral centroid of the whole face, the second-order bandwidth, the proportion of low-frequency and high-frequency energy, and the two types of asymmetry are assembled into energy window items and summarized in chronological order to form an energy window sequence.

[0026] This invention slices the vibration amplitude time series into fixed windows and extracts energy and spectral elements to construct an energy window sequence, so that the long time series is organized into a structured and statistical intermediate representation, balancing resolution and computational overhead, and providing a high signal-to-noise ratio input for subsequent coding and stage identification.

[0027] In this embodiment, the construction of the improved Transformer joint encoder specifically includes: The input layer and sequence organization of the improved Transformer joint encoder are constructed. For each time window of the energy window sequence, energy window items are extracted and assembled into feature vectors in a unified order. Position encoding vectors are generated for time identifiers and added to the feature vectors to form input entries. All entries are arranged into an input sequence according to time. At the same time, a corresponding index table and field mapping table are established to keep the input dimension consistent within the encoder. Set up a stable pole state space convolutional unit, which includes a pole parameter library, a sparse gate layer, a convolution kernel generator, and a convolution channel. By establishing a hidden state buffer and a linear readout layer, the absolute value of the eigenvalues ​​of the state update matrix is ​​defined to be less than one to ensure pole stability. Multiple sets of pole parameters are preset to cover different time constant ranges. The sparse gate layer is configured to selectively activate pole groups according to time position, generate corresponding convolution kernels, and process the input sequence in a time-invariant convolution manner inside the unit. An energy-conserving attention unit is set up, including a projection layer, a similarity channel, a mask generator, a budget register, a recalibrator, and a weighted convergence channel. Linear projection layers for query, key, and value are configured to generate vector features. A similarity calculation channel and an energy budget register are established. An attention mask is constructed to limit the aggregation range of windows in the same window and adjacent windows, while other positions are masked. A weighted recalibrator is established to normalize and scale the similarity results according to the energy budget. A weighted convergence channel is configured to linearly combine the value vectors according to the weights, ensuring consistency with the input sequence in terms of time index and feature dimension.

[0028] This invention introduces an improved Transformer joint encoder composed of a stable pole state space convolution unit and an energy-conserving attention unit connected in parallel. It simultaneously covers long and short time dependencies and suppresses non-physical aggregations with energy constraints, thereby improving the modeling stability and physical consistency of the entire night sequence.

[0029] In this embodiment, the output of the convolutional response sequence specifically includes: The encoder input sequence arranged in time is read from the input layer of the improved Transformer co-encoder. The input entries are obtained by adding the position information of the time marker and the energy window feature vector component by component. In the state space convolution unit of the stable pole, the hidden state buffer is initialized for each group of poles so that the absolute value of all eigenvalues ​​of the state update matrix is ​​less than one. The current input entries are sent to each pole group in sequence according to the time index and accumulated to the hidden state. The convolution response of the current pole group at the current time position is obtained by linear reading. At the same time, the convolution kernel generator generates an equivalent one-dimensional time-invariant convolution kernel according to the pole parameters and the preset kernel length. The convolution response of all pole groups at each time step is temporarily stored as a temporary sequence of convolution response of each pole group, and the corresponding time index is retained to align with subsequent gating and convergence. Based on the temporary sequence of convolutional responses of each pole group and the input entries at the same time position, the gating score of each time position for all pole groups is calculated in the sparse gating layer. The initial gating weights are obtained by exponential normalization. Only a preset number of pole groups are retained as the activation set from high to low. The initial weights of the selected pole groups are retained as effective weights. The weights of the non-selected pole groups are reset to zero. The sum of the effective weights at the same time position is constrained to not exceed one. The sparse gating weight sequence is output. At each time point, read the temporary sequence of convolutional responses of each pole group and the sparse gating weight sequence. Perform linear weighted summation on the convolutional responses of all activated pole groups at the time point according to the effective weights to obtain the output value of the state space convolutional branch of the stable pole. Arrange the output values ​​of all time points in chronological order to form a convolutional response sequence.

[0030] This invention employs multi-scale pole spectrum and sparse gating convolutional coding within stable pole state space convolutional units, activating different time constants as needed to match rhythmic changes, reducing redundant memory and numerical drift, and obtaining a smooth and timely convolutional response sequence.

[0031] In this embodiment, the output process of the context feature sequence specifically includes: Read the energy window sequence, calculate the short window energy budget for each time position, normalize it to the interval of zero to one, build an aggregateable index table containing only the index of the current window and adjacent windows, and send the input entries, energy budget and aggregateable index table into the energy conservation attention unit. Within the energy-conserving attention unit, query vector, key vector, and value vector are generated sequentially for each input entry at each time position. All three are obtained from their respective linear projection layers and have the same dimension. The unnormalized similarity score of the current time position to all candidate positions is calculated, and a binary mask is generated based on the aggregateable index table. Scores that do not belong to the current window and adjacent windows are marked as invalid. Row-wise exponential normalization is performed on the remaining valid scores to obtain the initial attention weights. The sum of the weights in the current row is calculated and compared with the corresponding energy budget to obtain a scaling factor not exceeding one. The initial attention weights are scaled item by item using the scaling factor to form a recalibrated weight distribution that satisfies the upper limit of the budget. Traverse the time position, multiply the recalibration weights with the corresponding value vectors one by one and sum them to obtain the context feature vector of the current position. Then, assemble the context feature vectors of all time positions into a context feature sequence in chronological order.

[0032] This invention inputs the energy window sequence into the energy-conserving attention unit and recalibrates it with the short-window energy budget and the adjacent window mask in the same window, thereby limiting the attention cross-window mis-aggregation and pseudo-energy amplification, and generating a context feature sequence that has both global dependence and energy constraints.

[0033] In this embodiment, the output of the temporal implicit representation sequence specifically includes: Based on the convolutional response sequence and the context feature sequence, the corresponding energy budget and recalibration coefficients are obtained simultaneously. After being aligned by time index, they are fused. The two vectors at the same time position are combined proportionally and weighted. The weights are automatically allocated according to the energy ratio of the two signals, with the higher energy vector having a slightly higher weight and the lower energy vector having a slightly lower weight, in order to obtain the initial fusion result while maintaining the time order. Energy adjustment and suppression control are applied to the initial fusion result. The amplitude of the fusion signal is scaled according to the energy budget value at each time position to keep the total energy within the budget range. The energy of the convolution branch and the attention branch at the same position are compared. When the energy of the attention branch is too high, its proportion is automatically reduced to prevent over-aggregation. At the same time, the output of the convolution branch is kept stable, and finally, the fusion sequence after energy balance is generated. Consistency correction is performed on the fused sequence. The deviation between the energy estimate and the corresponding budget at the current time point is detected. It is determined whether the deviation exceeds the allowable range. If it does, the amplitude of the current output is finely adjusted in the direction of the deviation. If it does not exceed the allowable range, it remains unchanged. The correction process is performed only once. The corrected results are output sequentially in time order to form a temporal implicit representation sequence.

[0034] This invention performs parallel fusion of convolutional response and contextual features, and combines energy prior and suppression feedback to complete a consistency correction, so that the temporal implicit representation has both stability and sensitivity characteristics, reduces jumps and trailing at stage switching points, and thus improves the overall staged robustness.

[0035] In this embodiment, the generation of the sleep cycle analysis report specifically includes: Read the temporal implicit representation sequence, analyze the feature changes of each time slice in chronological order, determine the sleep stage type corresponding to the current time slice, and record the time index and duration of the current stage. At the same time, calculate the matching confidence when each stage occurs to reflect the reliability of stage identification. Adjacent time slices belonging to the same stage are continuously merged to form stage segments. The start time, end time and duration of each segment are counted. The proportion of each stage in the total duration of sleep throughout the night is calculated. The transition relationship and number of transitions between stages are recorded. The distribution of major stages, including wakefulness, light sleep, deep sleep and REM sleep, is identified. Based on the distribution of major stages, a periodic curve of sleep stages changing over time is plotted with time as the horizontal axis. A band of confidence changes for each stage is superimposed to show stage stability and transition clarity. The proportion of each stage, average duration, and number of stage transitions are summarized. Combined with energy window features and time index, a whole-night sleep cycle analysis report is generated.

[0036] This invention generates a sleep cycle analysis report that includes stage proportions, durations, and transition statistics, and aligns it with a time index. It provides visualization and traceable markers, facilitating rapid assessment of treatment efficacy and the impact of daytime interventions in long-term home or ward settings, and reducing deployment and usage costs.

[0037] Example 1: To verify the feasibility of this invention in practice, it was applied to an overnight sleep cycle assessment scenario under non-contact vibration imaging conditions. The test environment was a natural sleep scenario with everyday bedding and stable lighting. A fixed-position video device was used for acquisition, with automatic gain and automatic white balance disabled to avoid introducing spurious vibrations. Subjects did not need to wear any hardware and went to bed as usual. The reference standard was a synchronously recorded polysomnography log, used as a control benchmark for stage labels, without altering the subjects' daily routines. This invention aims to solve the problems of spurious energy amplification, stage boundary drift, and cross-window aggregation caused by subtle fluctuations in lighting, camera noise, and local motion in traditional non-contact video methods over long overnight time series, providing a stable sleep stage sequence and confidence level consistent with the reference standard without increasing the subject's burden.

[0038] In this scenario, the invention first performs brightness stability and noise quality checks on the entire night video sequence, discarding unqualified frames while retaining timestamps to obtain a corrected video sequence for subsequent analysis. Then, target localization and cross-frame tracking are performed within the facial region. Micro-vibration intensity is extracted using adjacent frame difference and time accumulation, forming a vibration amplitude time series with time tags. Next, the time series is sliced ​​with a fixed window length and fixed step size to calculate energy, spectral centroid, dominant frequency, second-order bandwidth, and low-to-high frequency ratio, and this is combined with left-right asymmetry to construct an energy window sequence. These structured entries are then fed into an improved Trans... The former co-encoder employs a stable pole-state space convolutional unit that provides multi-timescale memory and numerical stability. An energy-conserving attention unit, constrained by short-window energy budgeting and adjacent window masks, limits cross-window attention aggregation, resulting in two intermediate representations. These two representations are fused in parallel at the same time index, with amplitude tuning and suppression based on energy priors. Finally, a consistency correction is performed, outputting a temporal implicit representation sequence. This sequence is then processed using thresholding and probabilistic strategies to obtain stage labels and confidence levels, generating a sleep cycle analysis report including stage proportions, durations, transition counts, and latency. The entire process is independent of wearable sensors, only provides offline comparison with reference standards, and does not interfere with the subject's sleep.

[0039] To quantify the beneficial effects of this invention, a representative multi-night dataset was selected for evaluation, covering subjects of different ages and with different sleep habits. Videos were collected each night under the same lighting intensity and camera position, with the video frame rate maintained near the recommended value within the resolvable range of the human head's micro-vibration frequency band, and the resolution within the engineering range that would not lose fine-grained motion information. During the quality control phase, the rate of change in brightness and intra-frame noise were recorded to ensure input stability. Two control methods were used: one was a common non-contact video scheme, using optical flow or simple frame difference energy as features and coupled with a traditional classifier; the other was a wearable triaxial accelerometer scheme. Both were compared with reference standards. Specific experimental data are shown in Table 1. Table 1. Comparison and Evaluation Table of the Method of the Invention and the Control Method

[0040] As shown in Table 1, compared to the conventional video method, this invention improves the accuracy of the entire nighttime sleep stage by approximately 8.5 percentage points, kappa by 0.16, reduces the average stage boundary error by approximately 48%, improves the transfer detection F1 to 0.74, reduces the absolute error of REM latency from 15.1 minutes to 7.9 minutes, significantly improves light robustness from 0.63 to 0.85, and reduces boundary jitter amplitude from 24.7 seconds to 11.2 seconds. Its overall performance is close to that of the wearable method. Although the inference time is slightly increased to 12.5 minutes, it achieves higher staging stability and interpretability under non-contact conditions. Overall, the results demonstrate that this invention can stably reconstruct the entire nighttime sleep structure in natural scenarios and improves the verifiability and clinical usability of the results through confidence level labeling and quality markers.

[0041] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for dynamic analysis of vibration image sleep cycles based on Transformer temporal modeling, characterized in that, Includes the following steps: The video sequence of the test object was collected overnight, and the brightness stability and noise were evaluated for each frame to obtain the corrected video sequence. Target region detection and cross-frame tracking are performed on the corrected video sequence to generate a vibration amplitude time series that reflects changes in micro-vibrations; The vibration amplitude time series is sliced ​​according to a fixed window length and a fixed step size to construct an energy window series; An improved Transformer joint encoder is constructed, comprising two substructures: a stable pole state space convolutional unit and an energy-conserving attention unit. A multi-scale pole spectrum array is established in the stable pole state space convolution unit, and the energy window sequence is convolutionally encoded to output the convolution response sequence; The energy window sequence is input into the energy conservation attention unit. In the energy conservation attention unit, an attention mask is constructed for each window based on the short window energy budget. Mask constraints and energy recalibration are performed to form a context feature sequence. The convolutional response sequence and the context feature sequence are fused in parallel, and forward energy prior transfer, backward inhibition feedback and consistency correction are performed to output the temporal hidden representation sequence. Calculate the sleep stage labels and confidence levels corresponding to each time slice of the time-series implicit representation sequence, and generate a sleep cycle analysis report.

2. The method for dynamic analysis of vibration image sleep cycles based on Transformer time-series modeling according to claim 1, characterized in that, The specific steps for obtaining the corrected video sequence include: A fixed-position camera was set up in the natural sleep environment of the subject, the light source was adjusted to make the brightness of the image uniform, the preview was turned on and the shooting parameters were locked, the time recording and calibration functions were enabled, and after eliminating reflections and obstructions, the night-long acquisition was started to generate the original video frame sequence and timestamp. During the acquisition process, quality inspection is performed on each frame of the original video frame sequence, the overall brightness change rate and the image noise level are calculated, and the inspection results and time information are recorded to form a set of qualified frames. The qualified frames are arranged in chronological order and the time intervals are checked. The sorted qualified frames are then packaged to obtain the corrected video sequence.

3. The method for dynamic analysis of vibration image sleep cycles based on Transformer time-series modeling according to claim 1, characterized in that, The process of generating the vibration amplitude time series specifically includes: Read the corrected video, locate the region of interest of the head and face in the first frame, divide the left and right sub-regions according to the midline of the bridge of the nose, start cross-frame tracking and update the region position and pixel mask frame by frame, calculate the overlap ratio with the previous frame, and output the region coordinates and mask sequence sorted by time. Within the defined region, the absolute value of the grayscale difference is calculated pixel by pixel in adjacent frames to obtain the vibration amplitude per unit time for the whole face, left side, and right side, respectively. At the same time, the center time of each window is recorded as a time label. The time stamps and the corresponding vibration amplitude values ​​per unit time for the whole face, left side, and right side are arranged in chronological order to form a vibration amplitude time series.

4. The method for dynamic analysis of vibration image sleep cycles based on Transformer time-series modeling according to claim 1, characterized in that, The construction of the energy window sequence specifically includes: Based on the vibration amplitude time series and time label, the vibration amplitude time series includes three vibration amplitudes per unit time for the whole face, left side and right side. A fixed window length and a fixed step size are set, and a continuous window is generated by sliding according to time. The three amplitudes and time points in each window are extracted, and the center time of the window is recorded as a time marker. For each window, calculate the energy values ​​of the whole face, left side, and right side separately. Based on the amplitude sequence of the whole face within the window, obtain the discrete spectrum energy distribution, and successively calculate the main frequency, the spectral centroid, and the second-order bandwidth. Calculate the energy ratio of low frequency and high frequency, and obtain the calculation results of the amplitude sequences of the left and right sides. Based on the calculation results of the left and right paths, the energy asymmetry and frequency asymmetry are calculated, and a quality mark is attached to the window to mask low-quality segments. The time stamp, the three energy values, the main frequency and spectral centroid of the whole face, the second-order bandwidth, the proportion of low-frequency and high-frequency energy, and the two types of asymmetry are assembled into energy window items and summarized in chronological order to form an energy window sequence.

5. The method for dynamic analysis of vibration image sleep cycles based on Transformer time-series modeling according to claim 1, characterized in that, The construction of the improved Transformer joint encoder specifically includes: The input layer and sequence organization of the improved Transformer joint encoder are constructed. For each time window of the energy window sequence, energy window items are extracted and assembled into feature vectors in a unified order. Position encoding vectors are generated for time identifiers and added to the feature vectors to form input entries. All entries are arranged into an input sequence according to time. At the same time, a corresponding index table and field mapping table are established. Set up stable pole state space convolutional units, define the eigenvalues ​​of the state update matrix by establishing a hidden state buffer and a linear readout layer, preset multiple sets of pole parameters, configure sparse gating layers to selectively activate pole groups according to time position, and generate corresponding convolutional kernels. Set up an energy-conserving attention unit, configure linear projection layers for query, key, and value, establish a similarity calculation channel and an energy budget register, construct an attention mask to limit the aggregation range of windows in the same window and adjacent windows, establish a weight recalibrator to normalize and scale the similarity results according to the energy budget, and configure a weighted convergence channel to linearly combine the value vectors according to the weights.

6. The method for dynamic analysis of vibration image sleep cycles based on Transformer time-series modeling according to claim 1, characterized in that, The output of the convolutional response sequence specifically includes: The encoder input sequence arranged in time is read from the input layer of the improved Transformer co-encoder. The input entries are obtained by adding the position information of the time marker and the energy window feature vector component by component. In the convolution unit of the stable pole state space, the hidden state buffer is initialized for each group of poles. The current input entries are sent to each pole group in sequence according to the time index and accumulated to the hidden state. The convolution response of the current pole group at the current time position is obtained by linear reading. At the same time, the convolution kernel generator generates an equivalent one-dimensional time-invariant convolution kernel according to the pole parameters and the preset kernel length. The convolution response of all pole groups at each time step is temporarily stored as a temporary sequence of convolution response of each pole group. Based on the temporary sequence of convolutional responses of each pole group and the input entries at the same time position, the gating score of each time position for all pole groups is calculated in the sparse gating layer. The initial gating weights are obtained by exponential normalization. Only a preset number of pole groups are retained as the activation set from high to low. The initial weights of the selected pole groups are retained as effective weights. The weights of the non-selected pole groups are reset to zero. The sum of the effective weights at the same time position is constrained to not exceed one. The sparse gating weight sequence is output. At each time point, read the temporary sequence of convolutional responses of each pole group and the sparse gating weight sequence. Perform linear weighted summation on the convolutional responses of all activated pole groups at the time point according to the effective weights to obtain the output value of the state space convolutional branch of the stable pole. Arrange the output values ​​of all time points in chronological order to form a convolutional response sequence.

7. The method for dynamic analysis of vibration image sleep cycles based on Transformer time-series modeling according to claim 1, characterized in that, The specific process of outputting the context feature sequence includes: Read the energy window sequence, calculate the short window energy budget for each time position, normalize it to the interval of zero to one, build an aggregateable index table containing only the index of the current window and adjacent windows, and send the input entries, energy budget and aggregateable index table into the energy conservation attention unit. Within the energy-conserving attention unit, query vectors, key vectors, and value vectors are generated sequentially for each input entry at each time position. The unnormalized similarity score of the current time position to all candidate positions is calculated, and a binary mask is generated based on the aggregateable index table. Row-wise exponential normalization is performed on the remaining valid scores to obtain the initial attention weights. The sum of the weights in the current row is calculated and compared with the corresponding energy budget to obtain the scaling factor. The initial attention weights are scaled item by item using the scaling factor to form a recalibrated weight distribution that satisfies the upper limit of the budget. Traverse the time position, multiply the recalibration weights with the corresponding value vectors one by one and sum them to obtain the context feature vector of the current position. Then, assemble the context feature vectors of all time positions into a context feature sequence in chronological order.

8. The method for dynamic analysis of vibration image sleep cycles based on Transformer time-series modeling according to claim 1, characterized in that, The output of the temporal implicit representation sequence specifically includes: Based on the convolutional response sequence and the context feature sequence, the corresponding energy budget and recalibration coefficients are obtained simultaneously. After being aligned by time index, they are fused. The two vectors at the same time position are combined proportionally with weights to obtain the initial fusion result. The initial fusion result is subjected to energy adjustment and suppression control. The fusion signal is scaled according to the energy budget value at each time position. The energy of the convolution branch and the attention branch at the same position are compared. Finally, the fusion sequence after energy balance is generated. Consistency correction is performed on the fused sequence to detect the deviation between the energy estimate and the corresponding budget at the current time point. The corrected results are output sequentially in chronological order to form a temporal implicit representation sequence.

9. The method for dynamic analysis of vibration image sleep cycles based on Transformer time-series modeling according to claim 1, characterized in that, The generation of the sleep cycle analysis report specifically includes: Read the time-series implicit representation sequence, analyze the feature changes of each time slice in chronological order, determine the sleep stage type corresponding to the current time slice, record the time index and duration of the current stage, and calculate the matching confidence when each stage occurs. Adjacent time slices belonging to the same stage are continuously merged to form stage segments. The start time, end time, and duration of each segment are counted to identify the main stage distribution. Based on the distribution of major sleep stages, a periodic curve of sleep stages changing over time is plotted with time as the horizontal axis. The proportion of each stage, the average duration, and the number of stage transitions are summarized to generate an overnight sleep cycle analysis report.

Images