A head motion pose prediction method, device and magnetic resonance system

Abstract

This invention relates to a method, apparatus, and magnetic resonance imaging (MRI) system for predicting head motion posture. The method includes: acquiring head motion posture data; constructing a state-space model of head motion based on the head motion posture data; and performing filtering estimation on the state-space model to obtain predicted values ​​of the head motion posture data. During the filtering estimation process of the state-space model, system noise and observation noise are corrected online in real time. This invention reduces the delay in the prediction process by introducing a Kalman prediction framework, effectively solving the problem of image latency.

Description

Technical Field

[0001] This invention relates to the field of motion prediction technology, and in particular to a method, device and magnetic resonance system for predicting head movement posture. Background Technology

[0002] Virtual display systems need to be able to respond to users' head movements in real time. Although users' head movements can be measured by various sensors, due to time constraints such as transmission and computation, head-mounted displays cannot immediately display the corresponding images to users, resulting in image delay. This delay causes a difference between visual motion perception and vestibular motion perception, which not only reduces the sense of presence of the virtual reality system, but also increases the burden on the brain and causes dizziness and nausea.

[0003] In the process of real-time, prospective motion correction, existing technologies continuously adjust the positioning of image slices to track the object in real time, thereby maintaining the relationship between spatial image information and the object's anatomical structure. This requires precise alignment of images from different time points in order to reliably quantify brain activity throughout the scanning process. However, due to reasons such as computing, communication, and systems, the image delay problem has not been effectively solved. Summary of the Invention

[0004] In view of this, it is necessary to provide a method, device and magnetic resonance system for predicting head motion posture in order to solve the problem of image latency in virtual / augmented reality.

[0005] To address the aforementioned problems, in a first aspect, the present invention provides a head movement posture prediction method, comprising:

[0006] Acquire head movement posture data;

[0007] A state-space model of head motion is constructed based on the head motion posture data.

[0008] The state space model is filtered and estimated to obtain the predicted value of the head motion posture data. During the filtering and estimation process of the state space model, system noise and observation noise are corrected online in real time.

[0009] Furthermore, acquiring head movement posture data includes:

[0010] Acquire multiple consecutive first head motion images captured by the image acquisition device;

[0011] Extract the pixel coordinates of preset feature points from the first head motion image;

[0012] Based on the preset spatial geometric constraints between the preset feature points, the pixel coordinates of the preset feature points are converted into the head motion posture data.

[0013] Furthermore, the state-space model of head motion includes the model's state equation and the model's observation equation; the construction of the state-space model of head motion based on the head motion posture data includes:

[0014] The translation, rotation angle, velocity, and acceleration of the head movement are used as state variables, and system noise is introduced to construct the state equation of the model.

[0015] The attitude measurement at the sampling time is used as the observation, and observation noise is introduced to construct the observation equation of the model.

[0016] Furthermore, the step of filtering and estimating the state-space model to obtain predicted values ​​of head motion posture data includes:

[0017] Historical state variables are obtained, and the state variables are updated according to the state equation of the model to obtain the initial predicted value of the head motion posture data.

[0018] Calculate the prior estimation error of the initial predicted value of the head motion posture data;

[0019] Calculate the Kalman gain based on the prior estimation error;

[0020] The initial prediction value is corrected based on the Kalman gain to estimate the predicted value of the head motion posture data.

[0021] Furthermore, the method also includes:

[0022] Calculate the posterior estimation error of the predicted values ​​of the head motion posture data;

[0023] The Kalman gain, prior estimation error, and posterior estimation error are iteratively optimized based on a preset Kalman filtering algorithm.

[0024] Furthermore, the real-time online correction of system noise and observation noise includes:

[0025] The measurement error is determined based on the state variables, observation matrix, and observations at the sampling time.

[0026] An intermediate variable is introduced based on the measurement error;

[0027] The system noise covariance is determined based on the intermediate variables and the Kalman gain.

[0028] The observation noise covariance is determined based on the intermediate variables, the observation matrix, and the Kalman gain.

[0029] The system noise and observation noise are corrected in real time online based on the system noise covariance and the observation noise covariance, respectively.

[0030] Furthermore, in the process of filtering and estimating the state-space model, the method further includes:

[0031] The predicted values ​​of multiple head motion posture data are used as historical state variables and re-inputted into the state space model to iteratively output multi-step predicted values ​​of head motion posture data.

[0032] Furthermore, after obtaining the predicted values ​​of the head motion posture data, the method further includes:

[0033] The second head motion image is obtained by rendering based on the predicted values ​​of the head motion posture data.

[0034] In a second aspect, the present invention also provides a head movement posture prediction device, comprising:

[0035] The acquisition module is used to acquire head motion posture data;

[0036] The construction module is used to construct a state-space model of head motion based on the head motion posture data;

[0037] The prediction module is used to perform filtering estimation on the state space model to obtain predicted values ​​of head motion posture data. During the filtering estimation process of the state space model, system noise and observation noise are corrected online in real time.

[0038] Thirdly, the present invention also provides a magnetic resonance system, comprising:

[0039] Image acquisition device, used to acquire images of the first head movement;

[0040] Head-mounted display device for displaying a second image of head movement;

[0041] A server is configured to communicate with the image acquisition device and the head-mounted display device, and includes a computer program for implementing the steps in the head motion posture prediction method described above when the computer program is executed.

[0042] The beneficial effects of using the above embodiments are:

[0043] This invention acquires head motion posture data, facilitating the monitoring and correction of head motion posture; it constructs a state space model of head motion points based on the head motion posture data, facilitating the analysis of actual head motion; finally, it performs filtering estimation on the state space model to obtain predicted values ​​of head motion posture data; this invention utilizes the continuous nature of head motion, predicts posture data with inherent delay based on historical head motion data, and then renders based on the predicted posture data, reducing screen latency. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating an embodiment of head motion posture prediction provided by the present invention;

[0045] Figure 2 This is a schematic diagram illustrating the principle of motion posture data measurement according to an embodiment of the present invention;

[0046] Figure 3 This is a flowchart illustrating a prediction process provided in one embodiment of the present invention;

[0047] Figure 4 This is a flowchart illustrating a head modeling motion prediction algorithm based on Kalman filtering provided in one embodiment of the present invention.

[0048] Figure 5 A schematic diagram of a structure of an embodiment of the head motion posture prediction device provided by the present invention;

[0049] Figure 6 This is a schematic diagram of the structure of a magnetic resonance system provided by the present invention. Detailed Implementation

[0050] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0051] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, "a plurality of" means two or more, unless otherwise explicitly specified. The reference to "embodiment" herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0052] The specific embodiments are described in detail below:

[0053] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of head motion posture prediction provided by the present invention. A specific embodiment of the present invention discloses a head motion posture prediction method, including:

[0054] Step S101: Acquire head motion posture data;

[0055] Step S102: Construct a state-space model of head motion based on head motion posture data;

[0056] Step S103: Filter and estimate the state space model to obtain the predicted value of the head motion posture data. During the filtering and estimation of the state space model, the system noise and observation noise are corrected online in real time.

[0057] The head motion posture data is obtained by mapping 2D images to 3D images and detecting the posture data in the 3D images. The head posture data in this invention mainly includes six degrees of freedom, such as the head translation vector (x, y, z) and the head rotation angle (α, β, γ). Specifically, image acquisition devices, such as industrial cameras and video cameras, can be used to acquire 2D head motion images, which are then transmitted to data processing devices such as computers. After processing, 3D head motion images are obtained. Feature points are then extracted from the 3D head motion images, and the head motion posture data is obtained based on the pose and position transformations of these feature points.

[0058] Understandably, after acquiring head pose data, the rendering of the image based on this data is not timely due to time constraints in transmission and computation, resulting in image delay. However, since head movement is continuous, the pose data after the inherent delay can be predicted based on historical head movement data. Then, rendering can be performed based on the predicted pose data, reducing image delay and enabling real-time tracking of head movement images.

[0059] This invention facilitates the monitoring and correction of head motion posture by acquiring head motion posture data; it constructs a state space model of head motion points based on the head motion posture data to facilitate the analysis of actual head motion; finally, it performs filtering estimation on the state space model to obtain the predicted value of head motion posture data, reducing tracking delay; and it improves the prediction accuracy of motion posture data by performing real-time online correction of system noise and observation noise.

[0060] It should be noted that in the prospective motion correction process of computer imaging and MRI, the object is typically tracked in real time by continuously adjusting the image slice positioning. However, due to computational, communication, and system limitations, image latency has not been effectively resolved. This invention proposes a head modeling motion prediction algorithm to predict the user's head pose for timely rendering of corresponding graphics. This minimizes the impact of motion latency on the immersive virtual reality system and reduces the latency introduced by the prospective motion correction calculation process.

[0061] In one embodiment of the present invention, acquiring head motion posture data includes:

[0062] Acquire multiple consecutive first head motion images captured by the image acquisition device;

[0063] Extract the pixel coordinates of preset feature points from the first head motion image;

[0064] Based on the preset spatial geometric constraints between preset feature points, the pixel coordinates of the preset feature points are converted into head motion posture data.

[0065] The head motion posture data includes six degrees of freedom for head movement, namely the head translation vector (x, y, z) and the head rotation angle (α, β, γ). Head posture data is acquired by mapping 2D images to 3D images to obtain the head orientation. The main parameters acquired include tilt...

[0066] The pitch angle (the angle of rotation around the X-axis), yaw angle (the angle of rotation around the Y-axis), and roll angle (the angle of rotation around the Z-axis), i.e., pitch, yaw, and roll, as well as the translation vectors along the X, Y, and Z axes.

[0067] For details on the measurement principles, please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram illustrating the principle of motion posture data measurement according to an embodiment of the present invention. The coordinate system mainly includes a world coordinate system, a camera coordinate system, and an image plane coordinate system. The world coordinate system can be fixed on the helmet during head movement, and its posture relative to the camera coordinate system is described by six parameters: translation vector (x, y, z) and rotation angle (α, β, γ). When the user's head drives the helmet in a motorized movement, an image acquisition device, such as a camera, continuously captures the first head motion image. Given the known spatial geometric constraints between various positioning feature points on the helmet, real-time measurement of head motion posture data (x, y, z, α, β, γ) can be achieved through a monocular vision posture measurement method based on point feature localization. The monocular vision posture measurement method will not be elaborated upon here.

[0068] Understandably, using the six degrees of freedom of the head as head motion posture data can better reflect the true motion state of the head; and by using a monocular vision posture measurement method based on point feature localization to measure the six degrees of freedom of the head, the measurement accuracy can be improved.

[0069] In one embodiment of the present invention, the state-space model of head motion includes the state equation of the model and the observation equation of the model; constructing the state-space model of head motion based on head motion posture data includes:

[0070] Using the translation, rotation angle, velocity, and acceleration of head movement as state variables, the state equation of the model is constructed as: X k =Φ·X k-1 +W k-1 .

[0071] Among them, X k Let X be the state variable at sampling time k, Φ be the state transition matrix, and X be the state variable at sampling time k. k W is the state variable at the previous sampling time. k-1 This is system noise;

[0072] Using the attitude measurement at sampling time k as the observation, the observation equation of the model is constructed: Z k =H·X k +V k .

[0073] Among them, Z k Let H be the observations at sampling time k, and V be the observation matrix. k To observe noise.

[0074] It is understandable that head movements are generally characterized by motorized motion, and because the camera's sampling period is very small, there will not be any drastic changes in motion within a single sampling period. Therefore, a uniformly accelerated motion model can be used to describe the actual head motion. Furthermore, in order to describe the head motion process more accurately, a state-space model of the head motion can be constructed using head pose data. This state-space model includes the model's state equations and the model's observation equations.

[0075] On the one hand, three translational and three rotational angles describing the head posture, along with their velocities and accelerations, can be chosen as the system state variables X. k = [x,x′,″,y,y′,″,,z′,″,,′,α″,,′,″,,′,″], and introduce system noise to construct the state equation of the model: X k =Φ·X k-1 +W k-1 .

[0076] Among them, X k Let X be the state variable at sampling time k, Φ be the state transition matrix, and X be the state variable at sampling time k. k W is the state variable at the previous sampling time. k-1 This represents system noise.

[0077] In the formula, Φ is the state transition matrix, which gives the system state variables X from sampling time k-1 to k. k-1 To X k The specific step of the transfer relationship, T is the sampling period. It is understandable that, since the motion model has a certain error compared to the actual motion, the model error is represented by the system noise W. k To compensate, the system noise reflects the deviation between the uniformly accelerated motion model and the actual motion of the helmet. Let it be zero-mean Gaussian noise, with a variance matrix of con(W). k )=Q k .

[0078] On the other hand, the attitude measurement at sampling time k can be used as an observation, and observation noise can be introduced to construct the observation equation of the model: Z k =H·X k +V k .

[0079] Among them, Z k Let H be the observations at sampling time k, and V be the observation matrix. k To observe noise.

[0080] In the formula, H is the observation matrix, which gives the system state variable X at sampling time k. k With observation Zk The transformation relationship between them, specifically,

[0081] Understandably, observation noise V is introduced because there is a certain error between the measured value and the actual value. k Compensation, the observation noise reflects the deviation between the measured and true values ​​of head movement posture, and is also set as zero-mean Gaussian noise, with its variance matrix being con(V k ) = R k Therefore, it can be approximated that the acceleration [x″,y″,z″,α″,β″,γ″] of the head posture translation and rotation angle remain constant within one sampling period.

[0082] In one embodiment of the present invention, please refer to Figure 3 , Figure 3 This is a flowchart illustrating a prediction process provided in one embodiment of the present invention.

[0083] Filtering and estimating the state-space model yields predicted values ​​for head motion pose data, including:

[0084] Step S301: Obtain historical state variables and update the state of historical state variables according to the state equation of the model to obtain the initial predicted value of head motion posture data.

[0085] First, it should be noted that this invention employs an unscented Kalman filter in the process of filtering and estimating the state-space model. The unscented Kalman filter is based on the unscented transform and uses a Kalman linear filtering framework to approximate the nonlinear distribution of the system state model through a sampling strategy.

[0086] Understandably, a complete Kalman filter cycle includes prediction and correction steps. In the prediction step, the model's state equations are used for advance prediction, where the historical state variables are historical head motion posture data, i.e., given previous state estimates. The initial predicted values ​​of the head motion pose data are the predicted prior state estimates. During the prediction process, the head pose position of the current frame can be predicted based on historical head motion pose data to obtain a priori state estimate. Specifically, the formula for updating the historical state variables according to the model's state equation to obtain the initial predicted value of the head motion pose data is as follows:

[0087] Step S302: Calculate the prior estimation error of the initial predicted values ​​of the head motion posture data.

[0088] Wherein, the prior estimation error is the prior state estimate. That is, the initial predicted value and the state variable X at sampling time kk Covariance between them, prior estimation error Specifically: In the formula, Φ is the state transition matrix, Φ T Let Φ be the transpose of matrix Φ, and Q be the covariance matrix of the system noise.

[0089] Step S303: Calculate the Kalman gain based on the prior estimation error.

[0090] Understandably, the Kalman gain needs to be calculated before making corrections. The Kalman gain represents the ratio of the predicted value to the measured value. Specifically, the formula for calculating the Kalman gain is: In the formula, H is the observation matrix, H T Let H be the transpose of matrix H, and R be the covariance matrix of the observation noise.

[0091] Step S304: Correct the initial prediction value based on the Kalman gain and estimate the predicted value of the head motion posture data.

[0092] Among them, the predicted values ​​of the head motion posture data are the corrected posterior state estimates. Specifically, based on the Kalman gain and the observation Z at sampling time k k For the initial predicted value, i.e. the prior state estimate. After making corrections, the posterior state estimate is obtained. The calculation formula is:

[0093] Understandably, Kalman filtering, through its prediction and correction steps, predicts and corrects head motion posture data to ensure the accuracy of the prediction.

[0094] In one embodiment of the present invention, during the process of filtering and estimating the state-space model using an unscented Kalman filter, the above method further includes:

[0095] Calculate the posterior estimation error of the predicted values ​​of head motion posture data.

[0096] The Kalman gain, prior estimation error, and posterior estimation error are iteratively optimized based on a pre-defined Kalman filtering algorithm.

[0097] It is understandable that the posterior estimation error is the state variable X at sampling time k. k With posterior state estimates That is, the covariance between predicted values, and the posterior estimation error. Specifically: In the formula, K k Let H be the Kalman gain and H be the observation matrix.

[0098] The preset Kalman filter algorithm can be an adaptive Kalman filter algorithm; the specific principles will not be elaborated here. It is understandable that by continuously optimizing the prior estimation error, posterior estimation error, and Kalman gain, the motion filtering at the next time step can be made more accurate.

[0099] In one embodiment of the present invention, real-time online correction of system noise and observation noise includes:

[0100] The measurement error is determined based on the state variables at the sampling time, the observation matrix, and the observations at the sampling time. k =Z k -H·X k ;

[0101] Introducing intermediate variables based on measurement error

[0102] Determining the system noise covariance based on intermediate variables and Kalman gain

[0103] The observation noise covariance R is determined based on intermediate variables, the observation matrix, and the Kalman gain. k+1 =C k -H·P k ·H T ;

[0104] The system noise and observation noise are corrected in real time online based on the system noise covariance and the observation noise covariance, respectively.

[0105] Understandably, in the process of establishing the state-space model, using uniformly accelerated motion to approximate the actual head motion will inevitably introduce model errors. During the iteration process, the system noise covariance Q, which describes the system noise, will also be affected. k and the observation noise covariance R that describes the observation noise k Q has a significant impact on the accuracy of attitude prediction and estimation. k and R k An inappropriate Q-factor selection can reduce system accuracy and, in severe cases, lead to filter instability or even divergence. However, due to the difficulty in fully understanding system characteristics, it is challenging to select a Q-factor that can accurately describe the statistical characteristics of system noise and observation noise. k and R k The initial value.

[0106] Adaptive filtering is a filtering method that can effectively suppress filter divergence. It can estimate and correct unknown or uncertain system model parameters and noise statistics online. Therefore, a noise statistical estimator can be introduced, specifically a noise statistical estimator based on the estimated residual to estimate Q in the recursive process. k and R kIt performs real-time, online corrections to ensure it accurately reflects the actual head movements.

[0107] The specific correction formula is as follows:

[0108] ∈ k =Z k -H·X k , R k+1 =C k -H·P k ·H T .

[0109] Where, ∈ k For measurement error, X k Let H be the state variable at sampling time k, and Z be the observation matrix. k For the observation at sampling time k, C k K is an intermediate variable introduced without any actual physical meaning. k This is the Kalman gain.

[0110] In one embodiment of the present invention, during the filtering estimation of the state-space model, the above method further includes:

[0111] The predicted values ​​of multiple head motion posture data are used as historical state variables and re-inputted into the state space model to iteratively output multi-step predicted values ​​of head motion posture data.

[0112] Understandably, under normal circumstances, we need to predict not only the motion prediction value at the next moment, but also the motion prediction values ​​at multiple future moments to achieve a given lead time. Each iteration of the Kalman filter generates a posterior state estimate. To obtain a result after N steps of prediction Posterior state estimation after each iteration By applying the state equation of the model N times forward propagation, the predicted result after time N can be obtained.

[0113] In one embodiment of the present invention, after obtaining the predicted value of the head motion posture data, the above method further includes:

[0114] The second head motion image is obtained by rendering based on the predicted values ​​of the head motion posture data.

[0115] Understandably, after obtaining the predicted values ​​of head motion posture data, it is necessary to render the corresponding graphics in a timely manner. Specifically, after obtaining the predicted values ​​of head motion posture data, such as head position information, it is transmitted to a remote server via the network. The remote rendering system presents complex graphics on a powerful remote server and then compresses and transmits the graphics to a less powerful client device via the network. This complex graphics are rendered using a fixed-delay six-degree-of-freedom rendering method based on the position information obtained from the server, thereby achieving real-time image rendering on the virtual display system and achieving accurate real-time tracking.

[0116] In real-time, prospective motion correction, such as the anticipated motion correction in parallel magnetic resonance imaging, cameras can be used to monitor and correct head movements. Leveraging the continuity of head motion and historical motion data, velocity and acceleration are calculated to predict positional information after inherent delays. To minimize the impact of motion image latency on immersive virtual reality systems and reduce the latency introduced by prospective motion correction calculations, this invention proposes a Kalman filter-based head modeling motion prediction algorithm to predict the user's head pose and render the corresponding graphics in a timely manner. Please refer to [link to relevant documentation]. Figure 4 , Figure 4 A flowchart illustrating a head modeling motion prediction algorithm based on Kalman filtering, provided in an embodiment of the present invention, includes:

[0117] Step S401: Obtain the six degrees of freedom of the head based on position tracking technology, and construct a state space model of head motion based on the six degrees of freedom of the head, wherein the state space model of head motion includes state equations and observation equations;

[0118] Step S402: Obtain historical state variables and make preliminary predictions on the historical state variables based on the state equation of the model to obtain prior state estimates;

[0119] Step S403: Calculate the prior estimation error of the prior state estimate;

[0120] Step S404: Calculate the Kalman gain based on the prior estimation error;

[0121] Step S405: Correct the prior state estimate based on the Kalman gain to obtain the posterior estimated filtered value;

[0122] Step S406: Calculate the posterior estimation error of the posterior estimated filter value;

[0123] Step S407: Correct the system noise in the state equation and the observation noise in the observation equation;

[0124] Step S408: Re-input multiple posterior estimated filtered values ​​as historical state variables into the state space model, and iteratively output multi-step predicted values ​​of head motion posture data.

[0125] Specifically, before making predictions, position tracking technology is used to obtain the six degrees of freedom of the head, namely the three translational and three rotational angles of the head pose, as the system's observation Z. k The observation equations for constructing the state-space model introduce measurement noise, denoted by V, due to the inherent error between measured and actual values. k To compensate, in order to more accurately describe the head movement process, three translational quantities and three rotational angles, along with their velocities and accelerations, are selected as the system state variables X. k Since the motion model has a certain error compared to the actual motion, the model error is introduced as process noise W. k To compensate.

[0126] Modeling head motion by considering the effects of velocity, acceleration, and noise, and based on the continuity of head motion and historical state variables... Preliminary predictions are made to obtain prior state estimates. Then, based on the state variable X at sampling time k... k Compared with prior state estimates Calculate the prior estimation error Calculate the Kalman gain based on the prior estimation error. The prior state estimate is corrected based on Kalman gain to obtain the posterior estimate filter value after motion, which is the predicted value of the head motion posture data. Based on the state variable X at sampling time k k With posterior estimated filter value Calculate the posterior estimation error Further optimization of the covariance matrix in the Kalman filter equation, namely the prior estimation error, the posterior estimation error, and the Kalman gain, results in more accurate motion filtering at the next time step.

[0127] Understandably, using uniformly accelerated motion to approximate the actual head motion during the establishment of the state-space model will inevitably introduce model errors; furthermore, due to the difficulty in fully understanding the system characteristics, it is difficult to select a system noise covariance Q that can describe the statistical characteristics of system noise and observation noise. k and observation noise covariance and R k Initial values. Therefore, this invention employs adaptive filtering to suppress filter divergence, which can estimate and correct unknown or uncertain system model parameters and noise statistics online. The specific correction formula is as follows: ∈ k =Z k-H·X k , R k+1 =C k -H·P k ·H T Wherein, ∈ k For measurement error, X k Let H be the state variable at sampling time k, and Z be the observation matrix. k For the observation at sampling time k, C k K is an intermediate variable introduced without any actual physical meaning. k This is the Kalman gain. A statistical estimator based on the estimated residual ground noise is introduced to adjust the Q value in the recursive process. k and R k It performs real-time, online corrections to ensure it accurately reflects the actual head movements.

[0128] Furthermore, it should be noted that each iteration of the Kalman filter generates a posterior estimate. To obtain a prediction of the result after N steps, the process model is applied N times to the posterior estimate after each iteration, i.e., the state equation is propagated forward, thus obtaining the prediction result after N time steps. Then, the predicted values ​​of the six degrees of freedom are sent in real time to the sequence of magnetic resonance images for gradient correction, reducing the impact of latency.

[0129] To better implement the head motion posture prediction method in the embodiments of the present invention, based on the head motion posture prediction method, please refer to the corresponding... Figure 5 , Figure 5 This is a schematic diagram of an embodiment of the head motion posture prediction device provided by the present invention. The embodiment of the present invention provides a head motion posture prediction device 500, comprising:

[0130] Module 501 is used to acquire head motion posture data;

[0131] Module 502 is used to construct a state-space model of head motion based on head motion posture data;

[0132] The prediction module 503 is used to perform filtering estimation on the state space model to obtain the predicted value of the head motion posture data. During the filtering estimation of the state space model, the system noise and observation noise are corrected online in real time.

[0133] It should be noted that the device 500 provided in the above embodiments can implement the technical solutions described in the above method embodiments. The specific implementation principles of the above modules or units can be found in the corresponding content in the above method embodiments, and will not be repeated here.

[0134] Based on the above-described head motion posture prediction method, this embodiment of the invention also provides a magnetic resonance system 600, including: an image acquisition device 601 for acquiring a first head motion image; a head-mounted display device 602 for displaying a second head motion image; and a server 603 for communicating with the image acquisition device 601 and the head-mounted display device 602, including a computer program for implementing the head motion posture prediction methods of the above embodiments when the computer program is executed.

[0135] In prospective motion correction using computed tomography and magnetic resonance imaging, the object is typically tracked in real time by continuously adjusting the positioning of image slices to maintain the relationship between spatial image information and the object's anatomical structure. This requires precise alignment of images from different time points to reliably quantify brain activity throughout the scanning process. However, due to computational, communication, and system issues, the problem of image delay has not been effectively resolved.

[0136] This invention proposes a head modeling motion prediction algorithm to predict the user's head pose for timely rendering of corresponding graphics. Specifically, an image acquisition device 601 collects the user's head pose data. Then, a server 603 activates a head motion pose prediction program, utilizing the continuity of head motion and historical head pose data to predict the position information after an inherent delay. The server 603 then renders complex graphics based on the predicted position information and transmits the compressed graphics to a less functional head-mounted display device 602 via a network.

[0137] Understandably, the complex graphics are rendered by server 603 based on the obtained six-DOF position information with a fixed delay after prediction. Therefore, image rendering can be performed on head-mounted display device 602 in real time, achieving accurate real-time tracking. This minimizes the impact of motion image latency on the immersive virtual reality system and reduces the latency caused by the forward motion correction calculation process.

[0138] Figure 6 Only some components of the magnetic resonance system 600 are shown. It is understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0139] In some embodiments of the present invention, the memory 604 may be an internal storage unit of the server 603, such as the hard disk or memory of the server 603, or it may be an external storage device of the server 603, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the server 603. Furthermore, the memory 604 is also used to store application software and various types of data installed on the server 603.

[0140] In some embodiments, processor 605 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 604 or process data, such as the head motion posture prediction method of the present invention.

[0141] In some embodiments, the display 606 in the head-mounted display device 602 can be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 606 is used to display information from the magnetic resonance system 600 and to display a visual user interface. The components 601-606 of the magnetic resonance system communicate with each other via a communication bus.

[0142] In some embodiments of the present invention, when the processor 605 executes the head motion posture prediction program in the memory 604, the following steps may be implemented:

[0143] Acquire head movement posture data;

[0144] Construct a state-space model of head motion based on head motion posture data;

[0145] The state-space model is filtered and estimated to obtain the predicted values ​​of head motion posture data. During the filtering and estimation process of the state-space model, system noise and observation noise are corrected online in real time.

[0146] It is understood that when the processor 605 executes the head motion posture prediction program in the memory 604, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0147] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0148] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for predicting head movement posture, characterized in that, include: Acquire head movement posture data; A state-space model of head motion is constructed based on the head motion posture data. The state-space model is filtered and estimated using an unscented Kalman filter to obtain predicted values ​​of head motion posture data. During the filtering and estimation process of the state-space model, real-time online corrections are performed for system noise and observation noise, including: The measurement error is determined based on the state variables, observation matrix, and observations at the sampling time. An intermediate variable is introduced based on the measurement error; The system noise covariance is determined based on the intermediate variables and the Kalman gain. The observation noise covariance is determined based on the intermediate variables, the observation matrix, and the Kalman gain. The system noise and observation noise are corrected in real time online based on the system noise covariance and the observation noise covariance, respectively.

2. The head movement posture prediction method according to claim 1, characterized in that, The acquisition of head movement posture data includes: Acquire multiple consecutive first head motion images captured by the image acquisition device; Extract the pixel coordinates of preset feature points from the first head motion image; Based on the preset spatial geometric constraints between the preset feature points, the pixel coordinates of the preset feature points are converted into the head motion posture data.

3. The head movement posture prediction method according to claim 1, characterized in that, The state-space model of head motion includes the state equation of the model and the observation equation of the model; the construction of the state-space model of head motion based on the head motion posture data includes: The translation, rotation angle, velocity, and acceleration of the head movement are used as state variables, and system noise is introduced to construct the state equation of the model. The attitude measurement at the sampling time is used as the observation, and observation noise is introduced to construct the observation equation of the model.

4. The head movement posture prediction method according to claim 1, characterized in that, The step of filtering and estimating the state-space model to obtain predicted values ​​of head motion posture data includes: Historical state variables are obtained, and the state variables are updated according to the state equation of the model to obtain the initial predicted value of the head motion posture data. Calculate the prior estimation error of the initial predicted value of the head motion posture data; Calculate the Kalman gain based on the prior estimation error; The initial prediction value is corrected based on the Kalman gain to estimate the predicted value of the head motion posture data.

5. The head movement posture prediction method according to claim 4, characterized in that, The method further includes: Calculate the posterior estimation error of the predicted values ​​of the head motion posture data; The Kalman gain, prior estimation error, and posterior estimation error are iteratively optimized based on a preset Kalman filtering algorithm.

6. The head movement posture prediction method according to claim 1, characterized in that, In the process of filtering and estimating the state-space model, the method further includes: The predicted values ​​of multiple head motion posture data are used as historical state variables and re-inputted into the state space model to iteratively output multi-step predicted values ​​of head motion posture data.

7. The head movement posture prediction method according to claim 1, characterized in that, After obtaining the predicted values ​​of head motion posture data, the method further includes: The second head motion image is obtained by rendering based on the predicted values ​​of the head motion posture data.

8. A head movement posture prediction device, characterized in that, include: The acquisition module is used to acquire head motion posture data; The construction module is used to construct a state-space model of head motion based on the head motion posture data; The prediction module is used to filter and estimate the state-space model using an unscented Kalman filter to obtain predicted values ​​of head motion posture data. During the filtering and estimation process of the state-space model, real-time online corrections are performed for system noise and observation noise, including: The measurement error is determined based on the state variables, observation matrix, and observations at the sampling time. An intermediate variable is introduced based on the measurement error; The system noise covariance is determined based on the intermediate variables and the Kalman gain. The observation noise covariance is determined based on the intermediate variables, the observation matrix, and the Kalman gain. The system noise and observation noise are corrected in real time online based on the system noise covariance and the observation noise covariance, respectively.

9. A magnetic resonance system, characterized in that, include: Image acquisition device, used to acquire images of the first head movement; Head-mounted display device for displaying a second image of head movement; A server is configured to communicate with the image acquisition device and the head-mounted display device, including a computer program for implementing the steps of the head motion posture prediction method according to any one of claims 1 to 7 when the computer program is executed.

Images