A wearable VR system for health education

Abstract

The present invention discloses a wearable VR system for health education, which relates to the field of virtual reality medical education technology and includes: a knowledge modeling unit for constructing a medical knowledge graph, a resource preprocessing unit for performing pre-allocation of rendering resources, an attention tracking unit for obtaining the line-of-sight focus, a motion prediction unit for collecting head poses, and an intelligent adaptation unit for realizing content scheduling. The present invention organizes teaching content by constructing a medical knowledge graph, combines the user's line-of-sight focus and head motion prediction for optimized resource allocation, and realizes the intelligent presentation of teaching scenarios and dynamic content scheduling.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual reality medical education, and particularly to a wearable VR system for health education. Background Art

[0002] The application of virtual reality technology in the field of medical health education has brought an innovative breakthrough to the traditional teaching mode. The current VR medical education systems mainly adopt static resource preloading and fixed rendering strategies, and display medical knowledge content through three-dimensional modeling and scene arrangement. Such systems generally use a knowledge organization method based on keyword matching to construct teaching content, use a simple rendering optimization strategy based on frustum culling, and obtain user attention information through traditional eye movement tracking algorithms. In terms of head movement prediction, it mainly relies on classical algorithms such as linear Kalman filtering for state estimation. The scheduling of teaching content usually adopts a preset branch selection structure, lacking the ability to adapt to the user's learning behavior in real time.

[0003] However, these technical solutions have obvious deficiencies in practical applications. First, the knowledge organization method based on keyword matching is difficult to accurately express the complex logical relationships between clinical diagnosis and treatment knowledge, affecting the coherence and professionalism of teaching content. Second, the simple rendering optimization strategy cannot effectively cope with the frequent perspective changes and content switching in the VR teaching scene, resulting in low rendering resource allocation efficiency. Third, the traditional eye movement tracking algorithm is prone to be affected by pupil imaging distortion and fixation edge region positioning deviation in the VR headset environment, reducing the accuracy of user attention analysis. In addition, the classical motion prediction algorithm is difficult to handle the non-linear characteristics of head movement and sensor noise, affecting the prediction accuracy of rendering optimization. Finally, the preset content scheduling structure lacks the ability to perceive and intelligently adapt to the user's learning state in real time, and cannot provide a personalized learning experience. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed.

[0005] Therefore, the present invention provides a wearable VR system for health education, which can solve the problems mentioned in the background art.

[0006] To solve the above technical problems, the present invention provides the following technical solution: A wearable VR system for health education, comprising: A knowledge modeling unit for constructing a medical knowledge graph; The knowledge modeling unit is connected to a resource preprocessing unit and an intelligent adaptation unit, and is used to transmit the medical knowledge graph to the resource preprocessing unit and respond to the teaching content retrieval request of the intelligent adaptation unit; A resource preprocessing unit for pre-allocating rendering resources for teaching scenarios according to the hierarchical association between the pathological knowledge nodes and the clinical diagnosis and treatment relationship edges in the medical knowledge graph; The resource preprocessing unit is connected to the intelligent adaptation unit and is used to receive the rendering strategy optimization instruction of the intelligent adaptation unit; An attention tracking unit for obtaining the user's line-of-sight focus and generating an attention heat map; The attention tracking unit is connected to the intelligent adaptation unit and is used to transmit the attention heat map to the intelligent adaptation unit; A motion prediction unit for collecting the user's head pose data and generating a head motion prediction trajectory; The motion prediction unit is connected to the intelligent adaptation unit and is used to transmit the head motion prediction trajectory to the intelligent adaptation unit; An intelligent adaptation unit for locating the currently concerned pathological knowledge node in the medical knowledge graph according to the attention heat map, dynamically optimizing the pre-allocation of rendering resources in the teaching scenario based on the head motion prediction trajectory, and selecting the teaching content for the next stage according to the hierarchical association of the pathological knowledge nodes.

[0007] As a preferred solution of the wearable VR system for health education according to the present invention, the knowledge modeling unit includes a knowledge extraction module, a relationship construction module, and a graph storage module; the knowledge extraction module is used to extract pathological knowledge nodes from medical textbooks; the relationship construction module is used to establish clinical diagnosis and treatment relationships between pathological knowledge nodes, specifically: using a multi-layer classification method to divide the node levels of pathological knowledge nodes; using a text semantic analysis method to extract context association features between pathological knowledge nodes; calculating the clinical relevance score between pathological knowledge nodes based on a deep neural network; the deep neural network adopts a bidirectional long short-term memory network structure, specifically by inputting the association feature vectors of two nodes into the deep neural network, extracting the clinical logical relationship between features through temporal dependence analysis, and using a sigmoid activation function to normalize the network output into a clinical relevance score within the range of 0 to 1; using a segmented mapping method to assign weight coefficients to the connections between pathological knowledge nodes; the segmented mapping method is based on the clinical relevance score, specifically when the clinical relevance score is less than the first preset threshold, the weight coefficient is assigned as A, when the clinical relevance score is between the first preset threshold and the second preset threshold, the weight coefficient is proportional to the score, and when the clinical relevance score is greater than the second preset threshold, the weight coefficient is assigned as B; using a graph structure quantization method to construct a multi-level knowledge structure; the graph storage module is connected to the relationship construction module and stores the constructed medical knowledge graph.

[0008] As a preferred solution of the wearable VR system for health education according to the present invention, wherein: the resource preprocessing unit includes a hierarchical analysis module, a resource allocation module, and a cache management module; the hierarchical analysis module performs quantitative calculation on the hierarchical association between the pathological knowledge nodes, specifically: calculating the out-degree and in-degree of the pathological knowledge nodes in the medical knowledge graph; the out-degree represents the number of the clinical diagnosis and treatment relationship edges pointing from the pathological knowledge node to other pathological knowledge nodes, and the in-degree represents the number of the clinical diagnosis and treatment relationship edges pointing to the pathological knowledge node; constructing the association propagation matrix of the pathological knowledge nodes, including the following steps: establishing an N×N square matrix; wherein, N is the total number of the pathological knowledge nodes in the medical knowledge graph, and the rows and columns of the square matrix respectively correspond to different pathological knowledge nodes; traversing each clinical diagnosis and treatment relationship edge in the medical knowledge graph, taking the starting node of the clinical diagnosis and treatment relationship edge as the row index of the square matrix and the ending node as the column index, and filling in the weight coefficient at the corresponding position to construct the initial association propagation matrix; wherein, if there is a direct connection between the pathological knowledge nodes, the value at the corresponding position of the association propagation matrix is the weight coefficient determined by the relationship construction module through the piecewise mapping method, and if there is no direct connection, it is zero; performing normalization processing on each row of the initial association propagation matrix so that the sum of the elements in each row is 1, and obtaining the normalized association propagation matrix M; generating the hierarchical importance score of the pathological knowledge nodes through iterative calculation, and after the iteration converges, taking the corresponding value in the importance vector obtained at the time of iterative convergence as the hierarchical importance score, and making the following adjustment according to the difference between the out-degree and in-degree of each pathological knowledge node: if the difference between the out-degree and in-degree of the pathological knowledge node is greater than half of the total connection number of the pathological knowledge node, then adjust the hierarchical importance score to the square of the current value; otherwise, keep the hierarchical importance score unchanged; the resource allocation module determines the priority of the pre-allocation of the rendering resources based on the edge computing device according to the hierarchical association, including the following steps: calculating the resource demand of the pathological knowledge nodes, and determining the resource allocation priority based on the hierarchical importance score; wherein, if the hierarchical importance score of the pathological knowledge node exceeds the average value of the layer where the pathological knowledge node is located, then improve the resource allocation priority of the pathological knowledge node; evaluating the computing power of the edge computing device and obtaining the available resource amount of the edge computing device; formulating the pre-allocation strategy of the rendering resources based on the resource allocation priority and the available resource amount; wherein, when the resource demand exceeds the available resource amount of the edge computing device, splitting the rendering task with a lower priority into multiple subtasks according to the task dependency relationship; the cache management module is connected to the resource allocation module and manages the storage space of the edge computing device.

[0009] As a preferred solution of the wearable VR system for health education according to the present invention, the attention tracking unit includes an image acquisition module, a pupil positioning module, and a heat map generation module; the image acquisition module acquires a sequence of user eye images at a preset frame rate through a binocular camera, and performs denoising and light compensation processing on the sequence of user eye images to obtain a line-of-sight image; the pupil positioning module performs pupil detection and corneal reflection point positioning on the line-of-sight image, and calculates the screen coordinates of the line-of-sight focus based on the relative positions of the pupil center and the corneal reflection point; wherein, if the pupil contour obtained by the pupil detection meets the ellipticity requirement and the brightness value of the corneal reflection point is greater than the average brightness value of the line-of-sight image, the screen coordinates of the line-of-sight focus are calculated using the positional relationship between the pupil center and the corneal reflection point; otherwise, the current line-of-sight image is marked as an invalid frame and re-acquisition is returned; wherein, the determination process for the pupil contour to meet the ellipticity requirement includes: extracting the contour point set of the pupil contour, calculating the minimum circumscribed rectangle of the contour point set, constructing a standard ellipse based on the major axis and minor axis of the minimum circumscribed rectangle, calculating the average distance deviation of the points in the contour point set from the standard ellipse, defining the ratio of the average distance deviation to the perimeter of the standard ellipse as the ellipticity fitting error, and determining whether the ellipticity fitting error is less than the ratio of the area to the perimeter of the pupil contour; the process of calculating the screen coordinates of the line-of-sight focus using the positional relationship between the pupil center and the corneal reflection point includes: establishing a polar coordinate system with the pupil center as the origin, calculating the polar coordinate parameters of the corneal reflection point relative to the pupil center, constructing a line-of-sight vector according to the polar coordinate parameters, and taking the intersection of the line-of-sight vector and the display plane as the screen coordinates of the line-of-sight focus; the heat map generation module performs spatio-temporal clustering analysis on the line-of-sight focus, and generates an attention heat map according to the residence time and spatial distribution characteristics of the line-of-sight focus; wherein, if the hierarchical importance score of the pathological knowledge node corresponding to the line-of-sight focus is higher than the average value of the nodes in the same layer, and the residence time of the line-of-sight focus in a certain area exceeds the average residence time of the adjacent areas, the heat value of the certain area in the attention heat map is set to the product of the residence time of the line-of-sight focus and the hierarchical importance score; otherwise, the heat value of the area is set to the residence time value of the line-of-sight focus.

[0010] As a preferred solution of the wearable VR system for health education according to the present invention, wherein: the motion prediction unit includes a data acquisition module, a state estimation module, and a trajectory prediction module; the data acquisition module acquires the head pose data of the user, performs low-pass filtering preprocessing on the head pose data of the user, and constructs a time series data sequence according to the sampling frequency of the sensor; the head pose data of the user includes three-axis acceleration, three-axis angular velocity, and three-axis magnetic field intensity; the state estimation module estimates the current motion state based on the recursive Bayesian prediction method, specifically by constructing a 16-dimensional state vector including position, velocity, acceleration, and attitude quaternion, establishing a non-linear state transition equation considering sensor noise, and using the particle filter algorithm for state estimation, and correcting the abnormal state through Kalman smoothing; the determination of the abnormal state includes state mutation determination, physical constraint violation determination, and sensor abnormality determination; the state mutation determination is realized by calculating the change amount of the state estimation value at adjacent moments and comparing it with the respective related thresholds, including position mutation, velocity mutation, acceleration mutation, and attitude angle mutation; the physical constraint violation determination includes speed overrun, acceleration overrun, and angular velocity overrun; the sensor abnormality is judged by the observation residual; when an abnormal state is detected, a fixed-interval Kalman smoother is used to correct the abnormal state; the trajectory prediction module is connected to the state estimation module and generates the predicted head motion trajectory.

[0011] As a preferred solution of the wearable VR system for health education according to the present invention, wherein: the execution process of the particle filter algorithm includes: sampling N particles from the state prior distribution, calculating the importance weight of each particle based on the observation equation, normalizing the importance weight, calculating the effective number of particles, and resampling when the effective number of particles is less than the effective particle number threshold, and finally obtaining the posterior distribution estimation of the state vector.

[0012] As a preferred solution of the wearable VR system for health education according to the present invention, wherein: the intelligent adaptation unit includes a node positioning module, an optimization strategy module, and a content scheduling module; the node positioning module determines the currently concerned pathological knowledge node in the medical knowledge graph; the optimization strategy module generates a dynamic optimization plan for the pre-allocation of rendering resources based on the deep reinforcement learning model; the content scheduling module executes the dynamic optimization plan and updates the teaching content of the next stage according to the hierarchical association of the pathological knowledge nodes.

[0013] To further solve the above technical problems, the present invention provides the following technical solutions: A method for a wearable VR system for health education, including: constructing a medical knowledge graph containing pathological knowledge nodes and clinical diagnosis and treatment relationship edges; the medical knowledge graph constructs the relationship between nodes through concept mapping; based on an edge computing device, pre-allocate rendering resources for teaching scenarios according to the hierarchical association between the pathological knowledge nodes and the clinical diagnosis and treatment relationship edges; obtain the user's line of sight focus through a binocular camera and generate an attention heat map; the attention heat map characterizes the attention distribution of the user to different regions in the teaching scenario; collect the user's head pose data based on a head pose sensor and generate a head movement prediction trajectory through recursive Bayesian prediction; the head movement prediction trajectory is used to estimate the user's field of view at the next moment; locate the currently concerned pathological knowledge node in the medical knowledge graph according to the attention heat map, dynamically optimize the pre-allocation of rendering resources in the teaching scenario based on the head movement prediction trajectory, and select the teaching content for the next stage according to the hierarchical association of the pathological knowledge nodes; the dynamic optimization uses a deep reinforcement learning model, and takes the attention heat map and the head movement prediction trajectory as state inputs.

[0014] A computer device includes a memory and a processor, the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the wearable VR system for health education as described above are implemented.

[0015] A computer-readable storage medium stores a computer program thereon, and is characterized in that when the computer program is executed by a processor, the steps of the wearable VR system for health education as described above are implemented.

[0016] Beneficial effects of the present invention: The present invention adopts a multi-head self-attention mechanism and a bidirectional long short-term memory network to construct a medical knowledge graph through a knowledge modeling unit, solving the problem that it is difficult to accurately grasp the logical relationship of clinical diagnosis and treatment knowledge based on traditional word frequency statistics; the resource preprocessing unit pre-allocates rendering resources based on graph structure features and hierarchical importance scores, overcoming the limitation that traditional resource scheduling methods are difficult to adapt to the dynamic switching requirements of VR teaching scenes; the attention tracking unit adopts pupil contour verification based on standard ellipse fitting error and line of sight vector mapping in polar coordinate system, effectively solving the problems of pupil imaging distortion and edge area gaze coordinate deviation in VR head display environment; the motion prediction unit combines recursive Bayesian framework and particle filter algorithm to predict head motion, improving the accuracy of nonlinear motion state estimation and abnormal state correction; the intelligent adaptation unit coordinately optimizes rendering resource allocation and teaching content scheduling through a deep reinforcement learning model, realizing scene adaptive presentation based on user attention and head motion. The system organically combines the structured expression of medical knowledge with the intelligent optimization of VR interactive experience, providing a new teaching solution for health education. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0018] Figure 1 This is a schematic diagram of the overall structure of a wearable VR system for health education proposed by the present invention;

[0019] Figure 2 This is an overall flow chart of the method of using a wearable VR system for health education proposed by the present invention. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0021] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0022] Example 1, referring to Figure 1 , which is an embodiment of the present invention, and provides a wearable VR system for health education.

[0023] Figure 1 Fig. shows the overall structural schematic diagram of a wearable VR system for health education, including the following:

[0024] A knowledge modeling unit for constructing a medical knowledge graph. The knowledge modeling unit is connected to the resource preprocessing unit and the intelligent adaptation unit, and is used to transmit the medical knowledge graph to the resource preprocessing unit and respond to the teaching content retrieval request of the intelligent adaptation unit.

[0025] A resource preprocessing unit for preallocating rendering resources for teaching scenarios based on edge computing devices according to the hierarchical association between pathological knowledge nodes and clinical diagnosis and treatment relationship edges in the medical knowledge graph. The resource preprocessing unit is connected to the intelligent adaptation unit and is used to receive the rendering strategy optimization instruction of the intelligent adaptation unit.

[0026] An attention tracking unit for obtaining the user's line of sight focus through a binocular camera and generating an attention heat map. The attention tracking unit is connected to the intelligent adaptation unit and is used to transmit the attention heat map to the intelligent adaptation unit.

[0027] A motion prediction unit for collecting the user's head pose data based on a head pose sensor and generating a head motion prediction trajectory through recursive Bayesian prediction. The motion prediction unit is connected to the intelligent adaptation unit and is used to transmit the head motion prediction trajectory to the intelligent adaptation unit.

[0028] An intelligent adaptation unit for locating the currently concerned pathological knowledge node in the medical knowledge graph according to the attention heat map, dynamically optimizing the preallocation of rendering resources in the teaching scenario based on the head motion prediction trajectory, and selecting the teaching content of the next stage according to the hierarchical association of the pathological knowledge nodes.

[0029] Specifically, as Figure 2 shown, it is the module structure diagram of each unit in a wearable VR system for health education provided by the present invention.

[0030] Among them, the knowledge modeling unit includes a knowledge extraction module, a relationship construction module, and a graph storage module. The knowledge extraction module is used to extract pathological knowledge nodes from medical textbooks, the relationship construction module is used to establish the clinical diagnosis and treatment relationships between pathological knowledge nodes, and the graph storage module is connected to the relationship construction module and stores the constructed medical knowledge graph.

[0031] The process of the knowledge extraction module extracting pathological knowledge nodes from medical textbooks includes the following steps:

[0032] Step 1: Scan the medical textbook text using a rule-based medical term recognition method;

[0033] Step 2: Use the domain dictionary matching method for the standard mapping of medical terms;

[0034] Step 3: Use the attribute annotation method to add labels to the pathological knowledge nodes.

[0035] Among them, the medical term recognition method is based on an expert annotation rule base and is used to recognize four types of key medical terms: disease names, symptom descriptions, diagnostic methods, and treatment plans. The domain dictionary matching method is based on character sequence matching, specifically including: establishing a medical term standard dictionary containing a comparison table of standard terms and their synonyms and near-synonyms, matching the recognized medical terms with the medical term standard dictionary, and generating a unique node identification code for the matched standard terms; the attribute labels include node type, node description, and node source.

[0036] The process of the relationship construction module establishing the clinical diagnosis and treatment relationship between pathological knowledge nodes includes the following steps:

[0037] Step 1: Use a multi-level classification method to divide the node levels of pathological knowledge nodes;

[0038] Step 2: Use the text semantic analysis method to extract the context association features between pathological knowledge nodes;

[0039] Step 3: Calculate the clinical relevance score between pathological knowledge nodes based on a deep neural network;

[0040] Step 4: Use the segmented mapping method to assign weight coefficients to the connections between pathological knowledge nodes;

[0041] Step 5: Use the graph structure quantization method to construct a multi-level knowledge structure.

[0042] Among them, the multi-level classification method divides the pathological knowledge nodes into four basic levels: disease level, symptom level, diagnosis level, and treatment level based on node attribute labels; the text semantic analysis method is based on the attention mechanism, specifically including: extracting the context window containing 10 sentences before and after the target node, using the multi-head self-attention mechanism to calculate the semantic correlation degree between each sentence in the window and the target node, and extracting the sentence with the highest attention weight as the association feature between nodes. The deep neural network adopts a bidirectional long short-term memory network structure, specifically including: inputting the association feature vectors of two nodes into the deep neural network, extracting the clinical logical relationship between features through temporal dependence analysis, and using the sigmoid activation function to normalize the network output into a clinical relevance score within the range of 0 to 1. The segmented mapping method is based on the clinical relevance score, specifically including: assigning a weight coefficient of A when the clinical relevance score is less than the first preset threshold, the weight coefficient is proportional to the score when the clinical relevance score is between the first preset threshold and the second preset threshold, and assigning a weight coefficient of B when the clinical relevance score is greater than the second preset threshold. The graph structure quantization method is based on weighted path analysis, specifically including: constructing an adjacency matrix according to the weight coefficients between nodes, calculating the shortest weighted path between nodes based on the adjacency matrix, and determining the hierarchical position relationship of nodes according to the weighted path, which is used to guide the resource preprocessing unit to allocate the priority of rendering resources. In this embodiment, the first preset threshold is set to 0.3, A is 0.1; the second preset threshold is set to 0.7, and B is 1.0.

[0043] Preferably, compared with the prior art, the relationship construction module of the present invention improves the construction quality of the medical knowledge graph by integrating the multi-head self-attention mechanism and the bidirectional long short-term memory network. Traditional methods usually rely on simple word frequency statistics or co-occurrence analysis of fixed windows to establish node associations, and it is difficult to accurately grasp the logical relationship of clinical diagnosis and treatment knowledge; while the attention mechanism introduced in the present invention can accurately capture the context semantic features, and the bidirectional LSTM structure can consider the forward and backward clinical logical dependencies at the same time, making the establishment of associations between nodes more in line with the laws of the medical profession. Especially in the weight assignment link, the three-stage mapping strategy based on dynamic thresholds designed in the present invention can more finely quantify the node association strength compared with the commonly used linear mapping or fixed threshold classification methods in the prior art. Experiments show that this multi-level relationship construction method improves the relationship accuracy of the knowledge graph by about 30%, and the automatic association accuracy of newly added nodes reaches more than 85%, providing a more reliable decision-making basis for the subsequent dynamic scheduling of VR teaching resources.

[0044] The process of the graph storage module storing the constructed medical knowledge graph includes the following steps:

[0045] Step 1, storing the multi-level knowledge structure by using the graph database storage method;

[0046] Step 2: Manage the access to knowledge nodes using the dynamic cache preheating method;

[0047] Among them, the graph database storage method is based on an index establishment mechanism, which specifically includes: establishing node indexes and relationship indexes, and constructing query interfaces based on node attributes, relationship types, and weight coefficients. The dynamic cache preheating method is based on access frequency statistics, which specifically includes: counting the number of accesses to nodes within the last 1 hour, loading the nodes and their associated nodes into memory when the number of node accesses exceeds 10 times per hour, and updating the cache content every 10 minutes to improve the data response speed during the switching of teaching scenarios.

[0048] It should be noted that after the construction of the medical knowledge graph, the knowledge modeling unit needs to transmit the medical knowledge graph to the resource processing unit through the data interaction interface. The specific process is as follows: First, the knowledge modeling unit serializes the constructed medical knowledge graph into a standard JSON format, which includes node data, relationship data, and weight information; then, the serialized data is transmitted to the data receiving module of the resource preprocessing unit in batches through the message queue mechanism; among them, the message queue adopts the publish-subscribe mode to ensure the reliability of data transmission; then, the data receiving module of the resource preprocessing unit deserializes the received JSON data and restores it to a graph structure; finally, based on the weight information in the graph structure, the resource preprocessing unit performs the initialization allocation of rendering resources and returns a confirmation message to the knowledge modeling unit after the data transmission is completed. This data transmission mechanism based on the message queue not only ensures the reliable transmission of large-scale knowledge graph data but also supports the incremental update of data.

[0049] Preferably, the knowledge modeling unit of the present invention organically combines the structured expression of medical textbook knowledge with the management of VR teaching resources, providing intelligent knowledge support for the wearable VR health education system. Compared with traditional knowledge management methods, this unit adopts semantic analysis based on the attention mechanism and a multi-level weight quantization strategy, which not only improves the construction accuracy of the medical knowledge graph, but more importantly, through deep cooperation with the resource preprocessing unit and the intelligent adaptation unit, realizes the accurate push of teaching content and the dynamic optimization of rendering resources. In the scenario of large-scale dynamic update of medical knowledge, the data interaction mechanism based on the message queue and the cache preheating strategy of the graph database in the knowledge modeling unit enable the system to efficiently process real-time data streams and ensure the smooth switching of VR teaching scenarios.

[0050] The resource preprocessing unit includes a hierarchical analysis module, a resource allocation module, and a cache management module. Among them, the hierarchical analysis module performs quantitative calculations on the hierarchical associations between pathological knowledge nodes, the resource allocation module determines the priority of rendering resource pre-allocation based on edge computing devices according to the hierarchical associations, and the cache management module is connected to the resource allocation module and manages the storage space of edge computing devices.

[0051] Specifically, the hierarchical analysis module performs quantitative calculations on the hierarchical associations between pathological knowledge nodes, which specifically include the following steps:

[0052] Step 1, calculate the out-degree and in-degree of the pathological knowledge nodes in the medical knowledge graph;

[0053] Step 2, construct the association propagation matrix of the pathological knowledge nodes;

[0054] Step 3, generate the hierarchical importance score of the pathological knowledge nodes.

[0055] Among them, the out-degree represents the number of clinical diagnosis and treatment relationship edges pointing from the pathological knowledge node to other pathological knowledge nodes, and the in-degree represents the number of clinical diagnosis and treatment relationship edges pointing to the pathological knowledge node.

[0056] In this embodiment, the association propagation matrix is constructed in the following way: First, establish an N×N square matrix, where N is the total number of pathological knowledge nodes in the medical knowledge graph, and the rows and columns of the square matrix correspond to different pathological knowledge nodes respectively; then traverse each clinical diagnosis and treatment relationship edge in the medical knowledge graph, use the starting node of the clinical diagnosis and treatment relationship edge as the row index of the square matrix, the ending node as the column index, fill in the weight coefficient at the corresponding position, and construct the initial association propagation matrix; finally, perform normalization processing on each row of the initial association propagation matrix so that the sum of the elements in each row is 1, and obtain the normalized association propagation matrix M.

[0057] Among them, filling in the weight coefficient at the corresponding position includes: If there is a direct connection between pathological knowledge nodes, the value at the corresponding position of the association propagation matrix is the weight coefficient determined by the relationship construction module through the piecewise mapping method, and if there is no direct connection, it is zero.

[0058] The hierarchical importance score is obtained through iterative calculation. The iterative calculation process of the hierarchical importance score is: First, assign the same initial importance value of 1.0 to each pathological knowledge node, and denote it as vector R 0 ; then in each round of iteration k, calculate the new importance vector through the following formula:

[0059] R k = βMR k-1 +(1 - β)R 0 ;

[0060] Among them, M is the normalized association propagation matrix; β is the importance attenuation factor, with a value of 0.85; k is the iteration round; R k is the node importance vector after the k-th round of iteration.

[0061] When the L1 norm of the difference between the importance vectors of two consecutive rounds of iteration satisfies the following conditions:

[0062] ‖R k -R k-1 ‖<∈;

[0063] It is considered that the iteration converges, where ∈ is the convergence threshold and its value is 0.001.

[0064] After the iteration converges, the value corresponding to the importance vector obtained at the time of iteration convergence is used as the hierarchical importance score, and the following adjustment is made according to the difference between the out-degree and in-degree of each pathological knowledge node: If the difference between the out-degree and in-degree of the pathological knowledge node is greater than half of the total number of connections of the pathological knowledge node, the hierarchical importance score is adjusted to the square of the current value; otherwise, the hierarchical importance score remains unchanged.

[0065] The process by which the resource allocation module determines the priority of pre-allocation of rendering resources based on edge computing devices according to hierarchical association includes the following steps:

[0066] Step 1, calculate the resource demand of the pathological knowledge node and determine the resource allocation priority based on the hierarchical importance score; among them, if the hierarchical importance score of the pathological knowledge node exceeds the average value of the layer where the pathological knowledge node is located, the resource allocation priority of the pathological knowledge node is increased;

[0067] Step 2, evaluate the computing power of the edge computing device and obtain the available resource amount of the edge computing device;

[0068] Step 3, formulate a pre-allocation strategy for rendering resources based on the resource allocation priority and the available resource amount; among them, when the resource demand exceeds the available resource amount of the edge computing device, the rendering tasks with lower priority are split into multiple subtasks according to the task dependency relationship.

[0069] Among them, the resource demand is calculated based on the hierarchical importance score and the rendering complexity of the pathological knowledge node. Specifically:

[0070] The hierarchical importance score of the pathological knowledge node is mapped to the interval [0, 1] through min-max normalization and denoted as w i , representing the normalized value of node importance; the rendering complexity parameter of the pathological knowledge node (including the sum of the quotient obtained by dividing the number of three-dimensional model patches by 10000 and the quotient obtained by dividing the texture resolution by 1024) is normalized and denoted as w c , representing the normalized value of rendering complexity; finally, the resource demand D is obtained by weighted summation:

[0071] D = αw i +(1 - α)w c ;

[0072] Among them, α is the importance weight coefficient and its value is 0.6.

[0073] The task dependencies in the rendering resource pre-allocation strategy are realized by constructing a directed acyclic graph: taking the pathological knowledge nodes as the vertices in the graph, taking the direction of the clinical diagnosis and treatment relationship edges as the dependency direction of task execution, and determining the execution order of the rendering tasks through topological sorting; when splitting the rendering tasks, the rendering integrity of the pathological knowledge nodes on the critical path is guaranteed first, and the rendering accuracy of the pathological knowledge nodes on the non-critical path can be reduced or the loading can be delayed. This fine resource scheduling mechanism ensures that the system can always maintain the optimal rendering effect and response speed under the condition of limited resources of the edge computing device.

[0074] The process of the cache management module managing the storage space of the edge computing device includes the following steps:

[0075] Step 1, dividing the storage space levels of the edge computing device;

[0076] Step 2, performing cache replacement operations;

[0077] Step 3, maintaining cache consistency.

[0078] Among them, the storage space is divided into a cache area, a main memory area, and an external memory area; the cache replacement is based on the least recently used principle. If the usage rate of the cache area exceeds the third preset threshold and the hierarchical importance score of the pathological knowledge node to be cached is higher than the average value of the cached nodes, the cache replacement operation is triggered; the consistency maintenance adopts the write-back policy to mark the updated pathological knowledge nodes.

[0079] It should be noted that in this embodiment, the cache replacement operation includes the following processing process: First, calculate the usage frequency scores of the cached nodes in the cache area. The calculation of the usage frequency scores is comprehensively evaluated through the following multiple dimensions: (1) Time dimension: Considering the time interval from the last access time of the node to the current time, as well as the access pattern and frequency distribution of the node within the recent time window; (2) Access dimension: Including statistical features such as the historical cumulative access times of the node, the length of the continuous access sequence, and the variance of the access interval; (3) Association dimension: Evaluating the association degree of the node with the current teaching scenario, including topological distance, semantic similarity and other features in the knowledge graph; (4) Resource dimension: Considering resource occupancy features such as the data size and rendering complexity of the node. The above multi-dimensional features are used to obtain the final usage frequency score through a machine learning model or a weighted calculation method; then, sort the cached nodes according to the usage frequency scores, select the node with the lowest score for elimination, and migrate its data to the main memory area; finally, load the data of the pathological knowledge node to be cached from the main memory area to the available space in the cache area.

[0080] For the cache management requirements in the VR medical education scenario, the third preset threshold is a key parameter determined by analyzing the multi-level storage structure of the edge computing device and the characteristics of medical teaching content switching. Since the knowledge nodes in the medical teaching scenario exhibit obvious locality access characteristics, that is, a group of related nodes will be repeatedly accessed within a specific time period, the setting of the third preset threshold needs to balance between cache utilization and replacement overhead. Statistical analysis based on large-scale teaching data shows that when the cache area capacity is set to 25% of the total memory of the edge computing device, setting the third preset threshold to 75% can effectively reduce the occurrence probability of cache thrashing while maintaining a high cache hit rate. In addition, considering the real-time requirement of scene switching in VR teaching, the setting of the third preset threshold also reserves sufficient cache space to handle sudden high-frequency access requests, enabling the system to have sufficient dynamic adaptability while maintaining stable performance.

[0081] Preferably, the present invention realizes pre-allocation of rendering resources for the teaching scenario through a resource allocation mechanism based on graph structure features in combination with an edge computing device. By analyzing the hierarchical association characteristics of pathological knowledge nodes, the pre-allocation of rendering resources is optimized, improving the rendering efficiency and response speed of the system.

[0082] The attention tracking unit includes an image acquisition module, a pupil positioning module, and a heat map generation module.

[0083] Specifically, the image acquisition module acquires a sequence of user eye images at a preset frame rate through a binocular camera, and performs denoising and light compensation processing on the sequence of user eye images to obtain a line-of-sight image.

[0084] The pupil positioning module performs pupil detection and corneal reflection point positioning on the line-of-sight image, and calculates the screen coordinates of the line-of-sight focus based on the relative positions of the pupil center and the corneal reflection point; among them, if the pupil contour obtained by pupil detection meets the ellipticity requirement and the brightness value of the corneal reflection point is greater than the average brightness value of the line-of-sight image, the position relationship between the pupil center and the corneal reflection point is used to calculate the screen coordinates of the line-of-sight focus; otherwise, the current line-of-sight image is marked as an invalid frame and returned for re-acquisition.

[0085] Among them, the determination process for the pupil contour to meet the ellipticity requirement includes: extracting the contour point set of the pupil contour, calculating the minimum circumscribed rectangle of the contour point set, constructing a standard ellipse based on the major axis and minor axis of the minimum circumscribed rectangle, calculating the average distance deviation of the points in the contour point set from the standard ellipse, defining the ratio of the average distance deviation to the perimeter of the standard ellipse as the ellipticity fitting error, and determining whether the ellipticity fitting error is less than the ratio of the area of the pupil contour to the perimeter.

[0086] The process of calculating the screen coordinates of the line-of-sight focus using the positional relationship between the pupil center and the corneal reflection point includes: establishing a polar coordinate system with the pupil center as the origin, calculating the polar coordinate parameters of the corneal reflection point relative to the pupil center, constructing a line-of-sight vector based on the polar coordinate parameters, and taking the intersection point of the line-of-sight vector and the display plane as the screen coordinates of the line-of-sight focus. Among them, the direction angle of the line-of-sight vector is a function of the azimuth angle of the polar coordinate parameters, and the elevation angle of the line-of-sight vector is a function of the radial distance of the polar coordinate parameters.

[0087] It should be noted that, based on the usage scenario of the wearable VR system for health education of the present invention, the pupil positioning module effectively solves the problem of pupil imaging distortion caused by learners frequently turning their heads to watch virtual teaching content during VR health education by introducing a pupil contour verification method based on the standard ellipse fitting error. This method uses the area perimeter ratio of the pupil contour as an adaptive reference standard, overcomes the technical defect that the traditional fixed threshold method is difficult to adapt to the pupil characteristics of different learners in the VR headset wearing state, and improves the accuracy of pupil detection during the interaction process of health education content.

[0088] Preferably, by using the line-of-sight vector mapping method in the polar coordinate system to calculate the screen coordinates of the line-of-sight focus, the pupil positioning module of the present invention solves the coordinate deviation problem generated by the traditional linear mapping method when gazing at the edge area of the virtual scene in the VR display environment. This method constructs a line-of-sight vector using the polar coordinate relationship between the pupil center and the corneal reflection point. Compared with the plane projection method in the prior art, it can more accurately track the actual fixation position of learners when watching health education content, providing reliable line-of-sight data support for the intelligent adaptation of subsequent teaching content.

[0089] Furthermore, the heat map generation module performs spatio-temporal clustering analysis on the line-of-sight focus and generates an attention heat map according to the dwell time and spatial distribution characteristics of the line-of-sight focus.

[0090] If the hierarchical importance score of the pathological knowledge node corresponding to the line-of-sight focus is higher than the average value of the nodes in the same layer, and the dwell time of the line-of-sight focus in this area exceeds the average dwell time of the adjacent areas, then the heat value of this area in the attention heat map is set to the product of the dwell time of the line-of-sight focus and the hierarchical importance score; otherwise, the heat value of this area is set to the dwell time value of the line-of-sight focus.

[0091] Specifically, the spatio-temporal clustering analysis process of the line-of-sight focus includes: constructing a time series sequence of the line-of-sight focus, calculating the Euclidean distance and time interval between adjacent line-of-sight foci, and assigning the line-of-sight foci with an Euclidean distance less than the spatial threshold and a time interval less than the time threshold to the same cluster. Among them, the spatial threshold is the minimum size of the display area boundary of the pathological knowledge node, and the time threshold is a multiple of the sampling period of the time series sequence of the line-of-sight focus.

[0092] The process of generating an attention heat map based on the dwell time and spatial distribution characteristics of the line-of-sight focus includes: taking the center point of the spatio-temporal clustering as the center of the heat distribution, mapping the number of line-of-sight foci of each cluster to the variance of the Gaussian distribution, mapping the sum of the time intervals of the line-of-sight foci within the cluster to the peak value of the Gaussian distribution, and superimposing the Gaussian distributions of all clusters to obtain the attention heat map, where the variance of the Gaussian distribution is proportional to the cluster range and the peak value is proportional to the dwell time.

[0093] Preferably, the attention tracking unit of the present invention effectively solves the technical problem that it is difficult for traditional VR teaching systems to accurately capture the attention distribution of learners on health knowledge content by analyzing the line-of-sight behavior characteristics of learners in the VR health education scenario in real time and combining spatio-temporal clustering analysis with the attention heat map. Based on the temporal clustering method of line-of-sight foci, this unit can adaptively adjust the clustering parameters, enabling the system to more accurately identify the key knowledge point staying patterns of learners in the virtual teaching environment and providing a quantitative basis for the dynamic optimization of teaching content. In addition, the attention tracking unit adopts a heat map generation method based on the superposition of Gaussian distributions to visualize the attention distribution of learners in the VR health education content, overcoming the attention evaluation bias caused by simply accumulating the dwell time in the prior art. Through the heat calculation method that combines the line-of-sight dwell time with the importance score of knowledge nodes, the system can more objectively reflect the cognitive investment degree of learners in health education content, providing a more teaching-actual decision-making basis for the personalized push and difficulty adjustment of subsequent teaching content.

[0094] The motion prediction unit includes a data acquisition module, a state estimation module, and a trajectory prediction module. Among them, the data acquisition module obtains the head pose data of the user, the state estimation module estimates the current motion state based on the recursive Bayesian prediction method, and the trajectory prediction module is connected to the state estimation module and generates a predicted head motion trajectory.

[0095] Specifically, the head pose data of the user includes three-axis acceleration, three-axis angular velocity, and three-axis magnetic field intensity; the data acquisition module performs low-pass filtering preprocessing on the head pose data of the user and constructs a time series data sequence according to the sampling frequency of the sensor.

[0096] The recursive Bayesian prediction method includes: constructing a state vector containing position (x, y, z), velocity (v x , v y , v z ), acceleration (a x , a y , a z ), and attitude quaternion (q 0 , q 1 , q 2 , q 3The 16 - dimensional state vector is used to establish a non - linear state transition equation X that takes sensor noise into account. t = f(X t-1 ) + w t , and the particle filter algorithm is used for state estimation, and the abnormal state is corrected by Kalman smoothing.

[0097] Among them, (x, y, z) is the three - dimensional position coordinate of the head in the world coordinate system, with the unit of meter; (v x , v y , v z ) is the velocity component of the head in three directions, with the unit of m / s; (a x , a y , a z ) is the acceleration component of the head in three directions, with the unit of m / s²; (q 0 , q 1 , q 2 , q 3 ) is the unit quaternion describing the head attitude, where q 0 is the scalar part, and (q 1 , q 2 , q 3 ) is the vector part; w t is the system process noise, which follows a Gaussian distribution with a mean of 0 and a covariance matrix of Q.

[0098] Specifically, the determination of the abnormal state includes the determination of state mutation, the determination of physical constraint violation, and the determination of sensor abnormality; the determination of state mutation is achieved by calculating the change amount of the state estimation value at adjacent times and comparing it with the respective relevant thresholds, including position mutation |p t - p t-1 | > λ p , velocity mutation |v t - v t-1 | > λ v , acceleration mutation |a t - a t-1 | > λ a and attitude angle mutation |θ t - θ t-1 | > λ θ ; the determination of physical constraint violation includes velocity over - limit |v t | > v max , acceleration over - limit |a t | > a max and angular velocity over - limit |ω t | > ω max ; sensor abnormality is determined by the observation residual |Z t - h(X t )| > λ zMake a judgment; when an abnormal state is detected, use a fixed-interval Kalman smoother to correct the abnormal state.

[0099] Among them, λ p is the position mutation threshold, set to 0.1 m; λ v is the velocity mutation threshold, set to 2.0 m / s; λ a is the acceleration mutation threshold, set to 20.0 m / s 2 ; λ θ is the attitude angle mutation threshold, set to 45°; v max is the maximum allowable velocity, set to 3.0 m / s; a max is the maximum allowable acceleration, set to 30.0 m / s 2 ; ω max is the maximum allowable angular velocity, set to 720° / s; λ z is the sensor observation residual threshold, set according to the sensor characteristics.

[0100] Among them, the non-linear state transition equation represents the time-series change relationship of position, velocity, acceleration, and attitude quaternion as: position update Velocity update v x,t = v x,t-1 + a x,t-1 Δt, acceleration update a x,t = a x,t-1 + ε a,t , attitude update Δt is the system sampling time interval, set to 0.005 s (200 Hz sampling rate); ε a,t is the acceleration random perturbation, obeying a Gaussian distribution with a mean of 0 and a standard deviation of 0.1 m / s 2 ; ω t is the angular velocity vector at the current moment, directly measured by the gyroscope; is the unit vector of the rotation axis, obtained by normalizing the angular velocity vector; is the quaternion multiplication operator.

[0101] The execution process of the particle filter algorithm includes: sampling N particles from the state prior distribution, calculating the importance weights of each particle based on the observation equation, normalizing the weights, calculating the number of effective particles, and resampling when the number of effective particles is less than the effective particle number threshold, and finally obtaining the posterior distribution estimate of the state vector. The effective particle number threshold is set to 50% of the total number of particles.

[0102] A trajectory prediction module, connected to the state estimation module, is used to generate a head motion prediction trajectory based on the state estimation result. Among them, the trajectory prediction process includes: establishing a trajectory prediction model based on the long short-term memory network, extracting the time-varying features of head motion, constructing a prediction strategy according to the periodicity and continuity constraints of the motion features, and gradually predicting the head position and attitude parameters at future moments in a sliding time window manner;

[0103] Among them, the periodicity and continuity constraints of the motion features include: extracting the main frequency components of head motion through Fourier analysis as network input features, introducing a velocity smoothness loss function to ensure trajectory continuity, setting the constraint ranges of acceleration and angular velocity according to the physiological limits of human head motion, and performing dynamic smoothing processing on the predicted trajectory.

[0104] Preferably, the motion prediction unit of the present invention realizes accurate prediction of the head motion of VR headset users by combining the recursive Bayesian framework and deep learning methods. Compared with traditional prediction methods, this unit has obvious advantages in modeling the non-linear features of the motion state and processing sensor noise, providing reliable motion prediction support for the rendering optimization of teaching scenarios. Experiments show that the average position error of this prediction method is controlled within 5 mm, and the attitude error is controlled within 2°, which can meet the real-time interaction requirements of the VR health education scenario.

[0105] The intelligent adaptation unit includes a node positioning module, an optimization strategy module, and a content scheduling module. Among them, the node positioning module determines the current concerned pathological knowledge node in the medical knowledge graph, the optimization strategy module generates a dynamic optimization plan for pre-allocation of rendering resources based on the deep reinforcement learning model, and the content scheduling module executes the dynamic optimization plan and updates the teaching content of the next stage according to the hierarchical association of the pathological knowledge nodes.

[0106] Specifically, the node positioning module is responsible for locating the currently concerned pathological knowledge nodes in the medical knowledge graph. By receiving the attention heat map output by the attention tracking unit, it extracts the user's visual focus area, determines the concerned area using the set heat map threshold, maps it to the surface of the 3D model in the scene to obtain the corresponding model part identifier, and finally retrieves the relevant pathological knowledge nodes in the medical knowledge graph constructed by the knowledge modeling unit according to this model part identifier. The optimization strategy module generates a rendering resource pre-allocation plan based on the head movement prediction trajectory output by the movement prediction unit, specifically including: converting the prediction trajectory into a sequence of key path points containing position and pose information, calculating the frustum and visible area corresponding to each path point, and based on the preset resource allocation strategy, assigning rendering priorities to the models and textures in the visible area, where the main concerned area (the part related to the current node) is given the highest priority and full details, the secondary concerned area (the part related to adjacent nodes) is given medium priority and reduced details, and the background area is given the lowest priority and lowest details. The content scheduling module is responsible for executing the resource pre-allocation plan and updating the teaching content, including adjusting the model detail level LOD and texture resolution according to the priority output by the optimization strategy module, preloading the resources that may be used in the next stage (including the detailed content of the current node, the content of adjacent nodes directly connected to the current node, and the basic background resources), and selecting the candidate teaching content for the next stage according to the out-edge relationship of the nodes in the knowledge graph. The key parameter settings in this embodiment include: the heat map threshold is 0.7 (the normalized heat value), the frustum parameter FOV = 90 degrees, the near plane = 0.1 meter, the far plane = 100 meters, the LOD level is 3 levels (high / medium / low precision), the texture resolution is original / one-half / one-quarter, and the preloading buffer size is 100MB. The intelligent adaptation unit in this embodiment realizes the intelligent pre-allocation of rendering resources in the teaching scene by combining the user's attention distribution and head movement prediction. This unit can accurately track the user's learning focus, preload the content that is about to enter the field of view, and dynamically adjust the rendering details according to the degree of concern, thereby improving the rendering efficiency while ensuring the visual quality. In addition, based on the content organization method of the knowledge graph, the connection of the teaching content is made more natural and smooth.

[0107] In summary, the present invention uses a multi-head self-attention mechanism and a bidirectional long short-term memory network to construct a medical knowledge graph through a knowledge modeling unit, solving the problem that it is difficult to accurately grasp the logical relationship of clinical diagnosis and treatment knowledge based on traditional word frequency statistics; the resource preprocessing unit pre-allocates rendering resources based on graph structure features and hierarchical importance scores, overcoming the limitation that traditional resource scheduling methods are difficult to adapt to the dynamic switching requirements of VR teaching scenes; the attention tracking unit uses pupil contour verification based on standard ellipse fitting error and line of sight vector mapping in polar coordinates to effectively solve the problems of pupil imaging distortion and edge area gaze coordinate deviation in VR head-mounted display environment; the motion prediction unit combines the recursive Bayesian framework and particle filter algorithm to predict head motion, improving the accuracy of nonlinear motion state estimation and abnormal state correction; the intelligent adaptation unit coordinately optimizes rendering resource allocation and teaching content scheduling through a deep reinforcement learning model, realizing scene adaptive presentation based on user attention and head motion. The system organically combines the structured expression of medical knowledge with the intelligent optimization of VR interactive experience, providing a new teaching solution for health education.

[0108] Example 2, reference Figure 2 , which is an embodiment of the present invention, based on the first embodiment, this embodiment further provides a method of using a wearable VR system for health education.

[0109] Figure 2 The overall process diagram of the method includes:

[0110] S1: Construct a medical knowledge graph containing pathological knowledge nodes and clinical diagnosis and treatment relationship edges; the medical knowledge graph constructs the relationship between nodes through concept mapping;

[0111] S2: Pre-allocate rendering resources for teaching scenarios based on edge computing devices according to the hierarchical association between pathological knowledge nodes and clinical diagnosis and treatment relationship edges;

[0112] S3: Obtain the user's sight focus through the binocular camera and generate an attention heat map; the attention heat map represents the distribution of the user's attention to different areas in the teaching scene;

[0113] S4: Collecting user head posture data based on the head posture sensor and generating head motion prediction trajectory through recursive Bayesian prediction; the head motion prediction trajectory is used to estimate the user's field of view at the next moment;

[0114] S5: Locate the currently concerned pathological knowledge nodes in the medical knowledge graph according to the attention heat map, dynamically optimize the pre-allocation of rendering resources in the teaching scenario based on the predicted head movement trajectory, and select the teaching content for the next stage according to the hierarchical association of the pathological knowledge nodes; the dynamic optimization uses a deep reinforcement learning model, taking the attention heat map and the predicted head movement trajectory as state inputs.

[0115] Embodiment 3 is an embodiment of the present invention. The difference from the previous embodiment is that if the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs and other various media that can store program codes.

[0116] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0117] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection (electronic device) having one or more wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing it in a suitable manner if necessary, and then storing it in a computer memory.

[0118] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above-described embodiments, the multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having suitable combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0119] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A wearable VR system for health education, characterized in that: include: Knowledge modeling unit, used to build medical knowledge graph; The knowledge modeling unit is connected to the resource preprocessing unit and the intelligent adaptation unit, and is used to transmit the medical knowledge graph to the resource preprocessing unit and respond to the teaching content retrieval request of the intelligent adaptation unit; A resource preprocessing unit, used for pre-allocating rendering resources for teaching scenes according to the hierarchical association between pathological knowledge nodes and clinical diagnosis and treatment relationship edges in the medical knowledge graph; The resource preprocessing unit is connected to the intelligent adaptation unit and is used to receive a rendering strategy optimization instruction from the intelligent adaptation unit; Attention tracking unit, used to obtain the user's visual focus and generate an attention heat map; The attention tracking unit is connected to the intelligent adaptation unit and is used to transmit the attention heat map to the intelligent adaptation unit; A motion prediction unit, used to collect user head posture data and generate a head motion prediction trajectory; the motion prediction unit is connected to the intelligent adaptation unit and is used to transmit the head motion prediction trajectory to the intelligent adaptation unit; An intelligent adaptation unit, configured to locate the pathological knowledge node of current concern in the medical knowledge graph according to the attention heat map, dynamically optimize the pre-allocation of rendering resources in the teaching scene based on the head movement prediction trajectory, and select the teaching content of the next stage according to the hierarchical association of the pathological knowledge nodes; The knowledge modeling unit includes a knowledge extraction module, a relationship construction module and a graph storage module; The knowledge extraction module is used to extract pathological knowledge nodes from medical textbooks; The relationship building module is used to establish the clinical diagnosis and treatment relationship between pathological knowledge nodes, specifically: A multi-layer classification method is used to divide the node levels of pathological knowledge nodes; The text semantic analysis method is used to extract the contextual association features between pathological knowledge nodes; Calculating clinical correlation scores between pathological knowledge nodes based on a deep neural network; the deep neural network adopts a bidirectional long short-term memory network structure, specifically inputting the associated feature vectors of two nodes into the deep neural network, extracting the clinical logical relationship between the features through temporal dependency analysis, and using a sigmoid activation function to normalize the network output to a clinical correlation score in the range of 0 to 1; A segmented mapping method is used to assign weight coefficients to connections between pathological knowledge nodes; the segmented mapping method is based on a clinical relevance score, specifically, when the clinical relevance score is less than a first preset threshold, a weight coefficient is assigned as A, when the clinical relevance score is between the first preset threshold and the second preset threshold, the weight coefficient is proportional to the score, and when the clinical relevance score is greater than the second preset threshold, a weight coefficient is assigned as B; Use graph structure quantification methods to build multi-level knowledge structures; The graph storage module is connected to the relationship construction module and stores the constructed medical knowledge graph.

2. The wearable VR system for health education according to claim 1, characterized in that: The resource preprocessing unit includes a hierarchical analysis module, a resource allocation module and a cache management module; The hierarchical analysis module performs quantitative calculation on the hierarchical associations between the pathological knowledge nodes, specifically: Calculate the out-degree and in-degree of the pathological knowledge node in the medical knowledge graph; the out-degree represents the number of clinical diagnosis and treatment relationship edges pointing from the pathological knowledge node to other pathological knowledge nodes, and the in-degree represents the number of clinical diagnosis and treatment relationship edges pointing to the pathological knowledge node; Constructing the association propagation matrix of the pathological knowledge node includes the following steps: Establish an N×N square matrix; wherein N is the total number of the pathological knowledge nodes in the medical knowledge graph, and the rows and columns of the square matrix correspond to different pathological knowledge nodes respectively; Traverse each of the clinical diagnosis and treatment relationship edges in the medical knowledge graph, use the starting node of the clinical diagnosis and treatment relationship edge as the row index of the square matrix, and the ending node as the column index, fill in the weight coefficient at the corresponding position, and construct an initial association propagation matrix; wherein, if there is a direct connection between the pathological knowledge nodes, the value of the corresponding position of the association propagation matrix is ​​the weight coefficient determined by the relationship construction module through the segmented mapping method, and if there is no direct connection, it is zero; Normalizing each row of the initial correlation propagation matrix so that the sum of the elements in each row is 1, thereby obtaining the normalized correlation propagation matrix M; Generate the hierarchical importance score of the pathological knowledge node through iterative calculation. After the iteration converges, use the corresponding value in the importance vector obtained when the iteration converges as the hierarchical importance score, and make the following adjustments according to the difference between the out-degree and the in-degree of each pathological knowledge node: if the difference between the out-degree and the in-degree of the pathological knowledge node is greater than half of the total number of connections of the pathological knowledge node, adjust the hierarchical importance score to the square of the current value; otherwise, keep the hierarchical importance score unchanged; The resource allocation module determines the priority of the rendering resource pre-allocation based on the edge computing device according to the hierarchical association, including the following steps: Calculating the resource demand of the pathology knowledge node, and determining the resource allocation priority based on the hierarchical importance score; wherein, if the hierarchical importance score of the pathology knowledge node exceeds the average value of the hierarchical level of the pathology knowledge node, then increasing the resource allocation priority of the pathology knowledge node; Evaluate the computing capability of the edge computing device and obtain the available resources of the edge computing device; Formulate the rendering resource pre-allocation strategy based on the resource allocation priority and the available resource amount; wherein, when the resource demand exceeds the available resource amount of the edge computing device, split the rendering task with a lower priority into multiple subtasks according to the task dependency; The cache management module is connected to the resource allocation module and manages the storage space of the edge computing device.

3. The wearable VR system for health education according to claim 2, characterized in that: The attention tracking unit includes an image acquisition module, a pupil positioning module and a heat map generation module; The image acquisition module acquires a user eye image sequence at a preset frame rate through a binocular camera, and performs denoising and illumination compensation processing on the user eye image sequence to obtain a sight line image; The pupil positioning module performs pupil detection and corneal reflection point positioning on the sight line image, and calculates the screen coordinates of the sight line focus based on the relative position of the pupil center and the corneal reflection point; wherein, if the pupil contour obtained by the pupil detection meets the ellipticity requirement and the brightness value of the corneal reflection point is greater than the average brightness value of the sight line image, the screen coordinates of the sight line focus are calculated using the positional relationship between the pupil center and the corneal reflection point; otherwise, the current sight line image is marked as an invalid frame and returned for re-collection; The process of determining whether the pupil contour meets the ellipticity requirement includes: extracting a contour point set of the pupil contour, calculating a minimum circumscribed rectangle of the contour point set, constructing a standard ellipse based on the major axis and the minor axis of the minimum circumscribed rectangle, calculating an average distance deviation from points in the contour point set to the standard ellipse, defining a ratio of the average distance deviation to the perimeter of the standard ellipse as an ellipticity fitting error, and determining whether the ellipticity fitting error is less than a ratio of the area to the perimeter of the pupil contour; The process of calculating the screen coordinates of the sight focus using the positional relationship between the pupil center and the corneal reflection point includes: establishing a polar coordinate system with the pupil center as the origin, calculating the polar coordinate parameters of the corneal reflection point relative to the pupil center, constructing a sight vector according to the polar coordinate parameters, and taking the intersection of the sight vector and the display plane as the screen coordinates of the sight focus; The heat map generation module performs spatiotemporal clustering analysis on the gaze focus, and generates an attention heat map according to the residence time and spatial distribution characteristics of the gaze focus; wherein, if the hierarchical importance score of the pathological knowledge node corresponding to the gaze focus is higher than the average value of the nodes at the same level, and the residence time of the gaze focus in a certain area exceeds the average residence time of adjacent areas, then in the attention heat map, the heat value of a certain area is set to the product of the gaze focus residence time and the hierarchical importance score; otherwise, the heat value of the area is set to the gaze focus residence time value.

4. The wearable VR system for health education according to claim 3, characterized in that: The motion prediction unit includes a data acquisition module, a state estimation module and a trajectory prediction module; The data acquisition module acquires the user's head posture data, performs low-pass filtering preprocessing on the user's head posture data, and constructs a time series data sequence according to the sampling frequency of the sensor; the user's head posture data includes three-axis acceleration, three-axis angular velocity and three-axis magnetic field intensity; The state estimation module estimates the current motion state based on the recursive Bayesian prediction method, specifically constructing a 16-dimensional state vector including position, velocity, acceleration and attitude quaternion, establishing a nonlinear state transfer equation considering sensor noise, using a particle filter algorithm for state estimation, and correcting abnormal states through Kalman smoothing; the determination of the abnormal state includes state mutation determination, physical constraint violation determination and sensor abnormality determination; the state mutation determination is realized by calculating the change in the state estimation value at adjacent moments and comparing it with the respective relevant thresholds, including position mutation, velocity mutation, acceleration mutation and attitude angle mutation; the physical constraint violation determination includes speed overrun, acceleration overrun and angular velocity overrun; the sensor abnormality is judged by observing the residual; when an abnormal state is detected, a fixed interval Kalman smoother is used to correct the abnormal state; The trajectory prediction module is connected to the state estimation module and generates the head movement prediction trajectory.

5. The wearable VR system for health education according to claim 4, characterized in that: The execution process of the particle filter algorithm includes: sampling N particles from the state prior distribution, calculating the importance weight of each particle based on the observation equation, normalizing the importance weight, calculating the effective particle number, resampling when the effective particle number is less than the effective particle number threshold, and finally obtaining the posterior distribution estimate of the state vector.

6. The wearable VR system for health education according to claim 5, characterized in that: The intelligent adaptation unit includes a node positioning module, an optimization strategy module and a content scheduling module; The node positioning module determines the pathology knowledge node of current concern in the medical knowledge graph; The optimization strategy module generates a dynamic optimization scheme for pre-allocation of rendering resources based on a deep reinforcement learning model; The content scheduling module executes the dynamic optimization scheme and updates the teaching content of the next stage according to the hierarchical association of the pathological knowledge nodes.

7. A method using the wearable VR system for health education as claimed in any one of claims 1 to 6, characterized in that: include, Constructing a medical knowledge graph including pathological knowledge nodes and clinical diagnosis and treatment relationship edges; the medical knowledge graph constructs the relationship between nodes through concept mapping; Pre-allocating rendering resources for teaching scenes based on edge computing devices according to the hierarchical association between the pathological knowledge nodes and the clinical diagnosis and treatment relationship edges; Use binocular cameras to obtain the user's visual focus and generate an attention heat map; The attention heat map represents the distribution of users' attention to different areas in the teaching scene; The head posture data of the user is collected based on the head posture sensor and a head movement prediction trajectory is generated through recursive Bayesian prediction; the head movement prediction trajectory is used to estimate the field of view range of the user at the next moment; Locating the pathological knowledge node of current concern in the medical knowledge graph according to the attention heat map, dynamically optimizing the pre-allocation of rendering resources in the teaching scene based on the head movement prediction trajectory, and selecting the teaching content of the next stage according to the hierarchical association of the pathological knowledge nodes; The dynamic optimization adopts a deep reinforcement learning model and takes the attention heat map and the head movement prediction trajectory as state inputs.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the wearable VR system for health education described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the wearable VR system for health education described in any one of claims 1 to 6 are implemented.

Images