Immersive virtual reality interaction method and device, electronic equipment and medium

Abstract

The invention provides an immersive virtual reality interaction method and device, and relates to the technical field of education, in particular to the technical fields of network analysis, computers, three-dimensional environments and the like. According to the specific implementation scheme, the method comprises the steps of creating an immersive virtual reality learning environment, and configuring environment attribute values of a plurality of virtual bodies in the immersive virtual reality learning environment; gaze data of the learning object under the environment attribute value are collected; constructing a gaze network based on the gaze data; and determining a learning result of the learning object based on the gaze network.

Description

Technical Field

[0001] The present invention relates to the field of educational technologies, and in particular to the fields of virtual reality, network analysis, computers, three-dimensional environments, etc., and particularly to an immersive virtual reality interaction method and apparatus, an electronic device, and a computer-readable storage medium. Background Art

[0002] From the applications of immersive virtual reality (IVR) in engineering education, military applications, and medical training to environmental education, as well as virtual field trips and scientific simulations in primary and secondary schools, immersive virtual reality and its environmental attribute values are becoming increasingly popular in training and education. Most educational IVR applications focus on experiential learning, especially simulating experiences that are difficult or impossible for learners to experience in real life. However, in addition to IVR simulations that take learners out of the learning environment, transforming the "typical" learning environment into an IVR learning environment is a promising approach in educational research and practice. Understanding the learning environment as various environments where learning can occur, IVR technology enables the creation of computer-generated simulated environments that allow for realistic perception and seemingly real interactions in an artificial and fully controllable virtual world. Therefore, the IVR learning environment provides a simulated learning environment for learners to experience in a manner similar to the learning environment in the real world; however, the IVR learning environment and the virtual characters included can be freely designed and fully controlled simultaneously to change their appearance, behavior, and interactions.

[0003] The classroom situation in the real world is complex and dynamic, and students' classroom learning is significantly influenced by numerous context- and peer-related factors. The learning environment (including virtual students) has been found to be related to students' achievements and academic trajectories, as well as their emotions and motivations during the learning process. Assuming that virtual students greatly shape the learning experience of students not only in the real-world classroom but also in the IVR classroom environment, it is therefore crucial to understand how to examine and utilize the corresponding peer effects by transforming "traditional" classmates into virtual virtual students (i.e., simulations of other human participants in the IVR classroom or computer-based simulated virtual students). Summary of the Invention

[0004] The present disclosure provides an immersive virtual reality interaction method and apparatus, an electronic device, and a computer-readable storage medium.

[0005] According to a first aspect, an immersive virtual reality interaction method is provided, the method comprising: creating an immersive virtual reality learning environment and configuring environmental attribute values of a plurality of virtual agents in the immersive virtual reality learning environment; collecting gaze data of a learning object under the environmental attribute values; constructing a gaze network based on the gaze data; and determining a learning result of the learning object based on the gaze network.

[0006] According to a second aspect, an immersive virtual reality interaction device is provided, the device comprising: a creation unit configured to create an immersive virtual reality learning environment and configure environmental attribute values of a plurality of virtual agents in the immersive virtual reality learning environment; a collection unit configured to collect gaze data of a learning object under the environmental attribute values; a construction unit configured to construct a gaze network based on the gaze data; and a determination unit configured to determine a learning result of the learning object based on the gaze network.

[0007] According to a third aspect, an electronic device is provided, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in any implementation manner of the first aspect.

[0008] According to a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, the computer instructions being used to cause a computer to execute the method described in any implementation manner of the first aspect.

[0009] The immersive virtual reality interaction method and device provided by the embodiments of the present disclosure, first, create an immersive virtual reality learning environment and configure environmental attribute values of a plurality of virtual agents in the immersive virtual reality learning environment; secondly, collect gaze data of a learning object under the environmental attribute values; then, construct a gaze network based on the gaze data; and finally, determine a learning result of the learning object based on the gaze network. By setting the environmental attribute values of the immersive virtual reality learning environment to determine the learning result, the reliability of the analysis of the learning result is improved; the best learning environment can be determined from the configured environmental attribute values, improving the learning experience of the learning object.

[0010] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0012] Figure 1It is a flowchart according to an embodiment of the immersive virtual reality interaction method of the present disclosure;

[0013] Figure 2 It is a schematic structural diagram among subjects in the immersive virtual reality learning environment of the present disclosure;

[0014] Figure 3 It is a schematic structural diagram according to an embodiment of the immersive virtual reality interaction device of the present disclosure;

[0015] Figure 4 It is a block diagram of an electronic device for implementing the immersive virtual reality interaction method of the embodiments of the present disclosure. Detailed implementation manners

[0016] Unless otherwise clearly stated, in the whole specification and claims, the term "comprising" or its variations such as "including" or "having" etc. will be understood to include the stated elements or components, without excluding other elements or other components.

[0017] The technical solutions of the present disclosure are illustrated by the following specific embodiments. It should be understood that one or more steps mentioned in the present disclosure do not exclude the existence of other methods and steps before and after the combined steps, or other methods and steps can be inserted between these clearly mentioned steps. It should also be understood that these examples are only used to illustrate the present disclosure and not to limit the scope of the present disclosure. Unless otherwise specified, the numbers of each method step are only for the purpose of identifying each method step, rather than limiting the arrangement order of each method or limiting the implementation scope of the present disclosure. The change or adjustment of their relative relationships can also be regarded as the implementable scope of the present disclosure under the condition of no substantial technical content change.

[0018] For the raw materials and instruments used in the embodiments, there is no specific limitation on their sources, and they can be purchased in the market or prepared according to the conventional methods well-known to those skilled in the art.

[0019] When designing an IVR classroom for educational research and practice, a central goal is to realistically simulate a classroom scenario including social counterparts, so as to use the IVR classroom as a tool to obtain data on the socially relevant processes of student learning in the classroom, and then more effectively deploy certain socially relevant configurations for more effective learning in the IVR classroom (for example, in a remote learning scenario or using virtual virtual students as teaching agents). In the context of peer effects, in particular, the perceived proximity or distance (i.e., similarity vs. dissimilarity) by virtual students is regarded as a key aspect of the impact of the social environment on students' learning experiences.

[0020] Although IVR learning environments have been increasingly used in educational research and practice, there is still a lack of systematic insights into how different configurations, especially in IVR learning environments, affect users' perceptions of the IVR environment and virtual social counterparts.

[0021] Aiming at the deficiencies in traditional technologies, the present disclosure proposes an immersive virtual reality interaction method that collects the fixation data of learning objects, analyzes the fixation network of the fixation data under environmental attribute values, and uses the fixation network to verify the learning results of the learning objects. Through the learning results, the learning experience of students in the IVR classroom is deeply understood. This method not only promotes the progress of educational technology but also creates a more efficient and meaningful learning environment between educators and learners. Figure 1 FIG. 100 shows a flowchart of an embodiment of the immersive virtual reality interaction method according to the present disclosure. The above immersive virtual reality interaction method includes the following steps:

[0022] Step 101, create an immersive virtual reality learning environment and configure the environmental attribute values of multiple virtual agents in the immersive virtual reality learning environment.

[0023] In this embodiment, the immersive virtual reality learning environment is a simulated learning environment similar to the learning environment experience in the real world realized through immersive virtual reality technology. The immersive virtual reality learning environment includes multiple virtual agents, where the virtual agents can be virtual characters, virtual objects, virtual animals, virtual teaching contents, etc. The environmental attribute value is the specific characteristic value of the social activity characteristics of the virtual agent, such as the spatial position, visual position, and behavioral characteristics between the virtual agent and other virtual agents. The virtual agents in the immersive virtual reality learning environment can be freely designed and controlled through the environmental attribute values to change the appearance, behavior, and interaction of the virtual agent.

[0024] In this embodiment, the obvious feature of the immersive virtual reality technology is that it uses a display device to enclose the vision and hearing of the learning object to generate virtual vision. At the same time, the immersive virtual reality technology uses a data glove to enclose the tactile channel of the learning object to generate virtual tactile sensations.

[0025] In this embodiment, the learning object is an object that conducts classroom learning in the immersive virtual reality learning environment. The learning object accesses the immersive virtual reality learning environment through a real device and a data glove and conducts real classroom learning in the virtual reality learning environment. In order to effectively collect the learning results of the learning object in the immersive virtual reality learning environment, the virtual agent configured with the environmental attribute value can be a virtual agent related to the learning object.

[0026] In this embodiment, the environmental attribute value is the value of the environmental attribute, and the environmental attribute includes: the position of the virtual entity in the immersive virtual reality learning environment, the visualization style of the virtual entity, and the action performance of the virtual entity. The environmental attribute value varies with different environmental attributes. For example, the environmental attribute value of the position of the virtual entity in the immersive virtual reality learning environment is to view virtual classmates and teaching content from the front row or the back row. The environmental attribute value of the visualization style of the virtual entity represents the style of the roles of virtual classmates and virtual teachers.

[0027] Step 102, collect the fixation data of the learning object under the environmental attribute value.

[0028] In this embodiment, the learning object is a real object learning in the immersive virtual reality learning environment, or an object and the corresponding device. The fixation behavior of the learning object in the immersive virtual reality learning environment allows the execution entity on which the immersive virtual reality interaction method runs to insight into how the learning object pays attention to the information presented to them in the immersive virtual reality learning environment. The fixation data is the information that the learning object pays attention to the virtual entity presented to itself, and the fixation data includes: the number of fixations, the number of saccades, and the fixation duration.

[0029] In this embodiment, the fixation data can be the data extracted from the eye movement tracking data collected by an eye tracker.

[0030] Step 103, construct a fixation network based on the fixation data.

[0031] In this embodiment, the fixation network is the information of the overall fixation activity and connectivity of the learning object to the objects of interest, or the overall distribution of the fixations between the objects of interest of the learning object. Among them, the objects of interest can be virtual entities or the content issued by virtual entities (such as teaching content). The information of the overall fixation activity and connectivity is used to characterize the degree of attention to different virtual entities, and the overall distribution of the fixations between the objects of interest can characterize how frequently the learning object moves between different virtual entities.

[0032] In this embodiment, a network analysis method is used to convert the fixation data into a fixation network. Among them, the network analysis method is based on the mathematical theory of graph theory. The network analysis method is a tool widely used in bioinformatics, mainly used to study the structure and function of biological networks. In bioinformatics, biological molecules such as genes and proteins can be used as nodes, and the interaction relationships between them can be used as edges to construct a biological network diagram. Through this network analysis method, the characteristics of the topological structure, function, and evolution of the network can be revealed, so as to understand the complex relationships in the biological system and reveal its potential biological significance.

[0033] In this embodiment, the fixation data is used to extract objects of interest, obtaining multiple objects of interest; the objects of interest are used as nodes, and the interactions or relationships between the objects of interest are used as edges to construct a fixation network.

[0034] Step 104: Based on the fixation network, determine the learning result of the learning object.

[0035] In this embodiment, the learning result is the result of the learning object learning in the immersive virtual reality learning environment, and the learning result includes the degree of interest of the learning object in the object of interest, the situational self-concept, and the learning performance. Among them, the degree of interest: the degree of interest of the student in the teaching content in the immersive virtual reality learning environment, which can be quantified through the real-time emotional feedback of the learning object (such as facial expression analysis). The situational self-concept is the self-evaluation of the learning object's own ability in the immersive virtual reality learning environment, for example, measured through a self-report scale (such as "I think I performed well in this class"). Learning performance: the test score of the student after participating in the corresponding teaching content in the immersive virtual reality learning environment (such as the score of the knowledge mastery test).

[0036] In this embodiment, the above step 104 includes: extracting the first objects of interest belonging to the teaching content in the fixation network, calculating the number of the first objects of interest, and based on this number, determining the degree of interest of the learning object in the objects of interest. Among them, the larger the value of the number, the higher the degree of interest.

[0037] The immersive virtual reality interaction method provided by the embodiments of the present disclosure, first, creates an immersive virtual reality learning environment and configures the environmental attribute values of multiple virtual subjects in the immersive virtual reality learning environment; secondly, collects the fixation data of the learning object under the environmental attribute values; then, based on the fixation data, constructs a fixation network; finally, based on the fixation network, determines the learning result of the learning object. By setting the environmental attribute values of the immersive virtual reality learning environment to determine the learning result, the reliability of the learning result analysis is improved; the best learning environment can be determined from the configured environmental attribute values, improving the learning experience of the learning object.

[0038] In some embodiments of the present disclosure, the above immersive virtual reality interaction method further includes: optimizing the environmental attribute values based on the learning result to obtain optimized attribute values.

[0039] In this alternative implementation, the environmental attribute values include: in response to the learning result indicating that the degree of interest of the learning object in the first teaching content is less than that of other objects of interest in the fixation network, increasing the number of appearances of the first teaching content in the immersive virtual reality learning environment, and adding the number of appearances to the environmental attribute values of the first teaching content to obtain the optimized attribute values.

[0040] The immersive virtual reality interaction method provided in this embodiment optimizes the environmental attribute values based on the learning results to obtain optimized attribute values, which can make the created immersive virtual reality learning environment more in line with the needs of the learning object and improve the learning experience and effect of the learning object.

[0041] In some alternative implementation manners of the present disclosure, the above environmental attribute values include: spatial position values, visualization style values, and action performance values; optimizing the environmental attribute values based on the learning results to obtain optimized attribute values includes: determining the interaction degree and participation degree of the learning object based on the learning results; in response to the interaction degree and participation degree not meeting the interaction participation conditions, adjusting any one of the spatial position values, visualization style values, and action performance values to obtain optimized attribute values.

[0042] In this alternative implementation manner, the interaction degree refers to the frequency of behavioral interactions (such as the number of times of raising hands, the number of times of dialogue responses) between the student and the virtual student or teacher, and the social attention of the gaze data (such as the gaze centrality to peers, the gaze connectivity between peers). The participation degree refers to the degree of concentration of the student on the teaching content (such as the total gaze time on the teaching content, the uniformity of gaze distribution).

[0043] In this alternative implementation manner, the interaction participation conditions may include interaction conditions and participation conditions, and the interaction conditions and participation conditions can be set based on requirements. Detecting the interaction conditions includes: detecting whether the interaction frequency between the learning object and the virtual subject is less than a preset threshold, or detecting whether the gaze centrality of the learning object is less than the experimental baseline value; if it is detected that the interaction frequency between the learning object and the virtual subject is less than the preset threshold (such as the number of times of raising hands per hour < 3 times), or the gaze centrality (Degree Centrality, DC) of the learning object to the virtual subject is less than the experimental baseline value (such as DC peer < 0.5), it is determined that the interaction conditions are not met.

[0044] Detecting the participation conditions includes: detecting whether the proportion of the gaze time of the learning object on the virtual subject is less than the proportion threshold, or detecting whether the uniformity of the gaze distribution of the learning object is greater than the critical value. If it is detected that the proportion of the gaze time of the learning object on the teaching content is less than the proportion threshold (the proportion threshold can be set according to development requirements, such as the proportion threshold is 40%), or the uniformity of the gaze distribution (such as the overall uniformity chi-square test statistic > the critical value), it is determined that the participation conditions are not met, indicating that the attention is overly dispersed.

[0045] In this alternative implementation, the adjustment of any one of the spatial position value, visualization style value, and action performance value in response to the non - satisfaction of the interaction and engagement conditions to obtain the optimized attribute value includes: in response to detecting that the student's interest, self - concept, and performance do not reach the preset goals (such as interest score < 4 / 5, test score < 60 points), or detecting that the interaction and engagement do not meet the expectations (such as DC < 0.3 not within the range of 0.5 - 0.8, chi - square value of fixation uniformity > critical value), the adjustment is triggered.

[0046] In this alternative implementation, the above - mentioned immersive virtual reality interaction method further includes: in response to detecting that the student's interest, self - concept, and performance all reach the preset goals (such as interest score ≥ 4 / 5, test score ≥ 80 points), and the interaction and engagement meet the expectations (such as DC of peers within the range of 0.5 - 0.8, chi - square value of fixation uniformity < critical value), the current environmental attribute value is maintained.

[0047] As Figure 2 shown, a 2 * 2 * 4 between - subject of the digital intelligent courseware learning platform is adopted, where the between - subject is a test - used immersive virtual reality learning environment. In this immersive virtual reality learning environment, three environmental attributes are investigated: spatial position, visualization style, and the performance behavior of the virtual subject; among them, changing the spatial position of the virtual subject in the between - subject is to place the seat (belonging to the virtual subject) representing the learning object in the front row or the back row. The front row refers to all position rankings adjacent to the last row. In the front row, when the learning object experiences the course, either it is in a position close to the teaching center, and there is only one row of students between the learning object and the teacher and the course content, or it is in a position in the back row of the learning environment, and there are all the virtual subjects of the whole class between the learning object and the teacher and the course content. Changing the visualization style of the virtual subject in the between - subject is to use the virtual subject such as stylized (such as Figure 2 the cartoon in) and non - stylized (such as Figure 2 the stylized / realistic in) to observe its impact on the attention of the learning object. The behavior of the virtual subject is pre - programmed in the between - subject (such as Figure 2 20%, 30%, 65%, and 80% raise their hands, where 20%, 30%, 65%, and 80% refer to the proportion of virtual subjects raising their hands in the between - subject) to simulate the classroom interaction frequency and mode of different virtual subjects.

[0048] In this alternative implementation, the configuration of the seat position is not limited to Figure 2In the two scenarios of the front row or the back row shown, although the front row and the back row are selected as typical comparison scenarios in the experimental design (for example, there is only one row of peers between the students in the front row and the teacher, and the students in the back row need to observe the teaching content through the entire class), such a choice is mainly to verify the impact of spatial proximity on the student attention network under controllable conditions. By adjusting the relative positions of the students in the virtual classroom and changing the visibility of the peers and teaching content within their line of sight, the effects of different spatial configurations on the learning experience are explored.

[0049] In actual practice, one of the advantages of the IVR learning environment is its high degree of customization. The IVR learning environment allows for flexible design of the seating layout according to teaching needs, such as adding middle rows, side rows, or even enabling dynamic position adjustment (such as allowing students to move freely in the virtual classroom). These extended configurations can all be incorporated into the solution of the present disclosure. For example, a three-row seating arrangement can be designed to compare the differences in the gaze patterns of students in the front row, middle row, and back row; or a circular seating layout can be simulated to observe the social interaction effects from different perspectives.

[0050] In this alternative implementation, the learning outcomes include: the interest level of the object of interest and the learning performance. In response to a low interest level and a low learning performance, it is determined that the interaction level and participation level of the learning object are low, and any one of the spatial position value, visualization style value, and action performance value is adjusted to obtain an optimized attribute value.

[0051] The method for optimizing the environmental attribute values provided by the present disclosure optimizes the interaction and participation of students by adjusting the spatial position value, visualization style value, and action performance value in the IVR learning environment. Through customized seating arrangements and simulation of virtual peer behaviors, a personalized learning experience is provided, thereby enhancing the learning outcomes and academic achievements of students.

[0052] In some alternative implementations of the present disclosure, the above environmental attribute values include: the spatial position value of the virtual entity, the visualization style value of the virtual entity, and the action performance value of the virtual entity. Creating an immersive virtual reality learning environment and configuring the environmental attribute values of multiple virtual entities in the immersive virtual reality learning environment includes: creating an immersive virtual reality learning environment including at least two virtual entities, where the virtual entities include: a virtual teacher and at least one virtual student; configuring corresponding spatial position values for the positions between any two virtual entities; and configuring corresponding visualization style values and action performance values for each virtual entity.

[0053] In this alternative implementation, the environmental attributes in the immersive virtual reality learning environment include: spatial location, visualization style, and action performance. The spatial location value is the seating arrangement and position of the learning object and the virtual subject in the immersive virtual reality learning environment. The visualization style is the style of the virtual subject in the immersive virtual reality learning environment. The action performance is the interaction behavior of the virtual subject alone or with other virtual subjects in the immersive virtual reality learning environment. The environmental attribute values include: spatial location value, visualization style value, and action performance value.

[0054] In this alternative implementation, the environmental attribute values of the immersive virtual reality learning environment play an important role in the learning process of the learning object. Through the environmental attribute values, the key information that affects the learning results of students in the immersive virtual reality learning environment can be effectively determined, providing an effective acquisition basis for obtaining the fixation data of the learning object.

[0055] In some alternative implementations of the present disclosure, constructing a fixation network based on the fixation data includes: preprocessing the fixation data to obtain preprocessed data; clustering the preprocessed data to determine the attention data of the learning object; and obtaining a fixation network based on the attention data.

[0056] In this alternative implementation, the fixation data can be data related to the attention of the learning object. For example, if the fixation data is eye movement tracking data, the fixation data acquisition can be achieved through an eye movement tracking sensor built in the device of the immersive virtual reality learning environment. The fixation data includes: the fixation point coordinates, the line-of-sight direction vector, and the time stamp of the learning object.

[0057] In this alternative implementation, data preprocessing includes: preprocessing the fixation data, including data cleaning and standardization. For example: removing noise and invalid data points in the fixation data, converting data with different coordinate systems in the fixation data to the same coordinate system, and smoothing the data to reduce eye movement tracking noise.

[0058] In this alternative implementation, the attention data is the data that the learning object mainly focuses on. The line-of-sight clustering algorithm such as K-means or DBSCAN can be used to cluster the preprocessed data to determine the attention data of the learning object. The attention data includes the attention area or fixation object of the learning object, and both the attention area and the fixation object are virtual subjects.

[0059] In this alternative implementation, the attention network is the overall network of the attention allocation and transfer of the learning object to different virtual subjects (such as peers, teachers, teaching content, etc.) in the IVR classroom. This concept covers the visual attention distribution of the learning object in the classroom and how they switch attention between different points of interest.

[0060] In this alternative implementation, the "gaze network" is used to materialize and quantify the "attention network". Through eye-tracking technology, it can record the attention data of the learning object, and these attention data reflect the actual visual attention distribution of the learning object in the classroom. Nodes represent different points of interest (i.e., virtual entities gazed at by the learning object), and edges represent the gaze transitions of the learning object between these points of interest. The "gaze network" is a specific implementation and quantification tool for the "attention network". It can more precisely analyze and understand the attention distribution and transfer patterns of the learning object in the IVR classroom. By constructing and analyzing the gaze network, detailed information about the attention network of the learning object can be obtained, thereby further exploring the relationship between it and the learning experience.

[0061] The method for constructing a gaze network provided in this alternative implementation preprocesses the gaze data to obtain preprocessed data; clusters the preprocessed data to determine the attention data of the learning object; and obtains the gaze network based on the attention data. Thus, the gaze network is obtained through the attention data, improving the reliability of the obtained gaze network.

[0062] In some alternative implementations of the present disclosure, obtaining the gaze network from the above-mentioned attention data includes: constructing an initial attention network of the learning object, where nodes in the attention network represent the learning object and the weight of the edge represents the attention relationship between the learning objects; traversing the attention data of the learning object and adding corresponding weights in the initial attention network; and using a graph theory analysis algorithm to evaluate the influence and status of each learning object in the initial attention network to obtain the gaze network.

[0063] Assume gaze points is a two-dimensional array of gaze data, as shown in Equation (1).

[0064] gaze points = np.array([...]) (1)

[0065] Using the K-means algorithm for clustering, assuming that the number of clusters K is predefined, the clustering result as shown in Equation (2) can be obtained.

[0066] k means = K Means (n clusters = K).fit(gaze points ) (2)

[0067] In this alternative implementation, graph theory can be used to model the attention network among students. The attention network is regarded as the gaze network, where the nodes represent the virtual entities in the attention data, and the weights of the edges represent the attention relationships between the virtual entities. If the line of sight of a learning object is often near a virtual entity, an edge is added between them. By traversing the gaze data of all virtual entities and comparing the attention data of each pair of virtual entities, if there is mutual attention, the weight of the edge is increased.

[0068] The specific process of the gaze network is as follows: Create an undirected graph as shown in Equation (3).

[0069] G = nx.Graph() (3)

[0070] Add nodes to the undirected graph (assuming there are N virtual entities), as shown in Equation (4).

[0071] G = add nodesfrom (range(N)) (4)

[0072] Use graph theory analysis methods (such as PageRank or centrality analysis) to evaluate the influence and status of different nodes in the attention network, as shown in Equation (5), and calculate the influence and status Page rank .

[0073] page rank = nx.pagerank(G, weight='weight') (5)

[0074] The method for obtaining the gaze network provided in this alternative implementation determines the gaze-based attention network of students in the IVR classroom under different configuration conditions, in order to obtain an in-depth and objectively measurable gaze network, understand how virtual entities in different IVR classrooms affect the way students focus on information in the IVR classroom scenario, and improve the reliability of the immersive virtual reality interaction method.

[0075] In some alternative implementations of the present disclosure, determining the learning result of a learning object based on the above-mentioned gaze network includes: determining structural variables based on the gaze network; inputting the structural variables into a pre-trained graph neural network model to obtain the learning result variables output by the graph neural network model; and determining the learning result of the learning object based on the learning result variables.

[0076] In this alternative implementation, the gaze network is an analysis tool through which the attention distribution and learning experience of students in the IVR classroom can be understood. This gaze network is constructed based on the gaze data of students, where the nodes represent different points of interest (virtual entities such as virtual peers, virtual teachers, and virtual teaching content), and the edges represent the gaze transfer of the learning object between these points of interest.

[0077] In this alternative implementation, the structural variable is a variable that characterizes the structure of the fixation network. This variable is used to quantify and analyze specific metrics of the fixation network. According to the specific structure of the fixation network, the structural variable can be divided into multiple different categories, and each category can describe the distribution of the learning object's attention in the immersive virtual reality learning environment from different perspectives.

[0078] In this alternative implementation, the graph neural network model is used to characterize the relationship between the structural variable of the fixation network and the learning outcome variable. The graph neural network model includes: multiple graph convolution layers (Graph Convolution Layer), a global pooling layer, and a fully connected layer; among them, the graph convolution layer is used to obtain the feature representation of nodes. Assume H (l) is the node feature matrix of the l-th layer, where each row represents the feature vector of a node. Through the graph convolution operation, the feature representation of the (l + 1)-th layer is obtained, as shown in Equation (6):

[0079] H (l+1) = σ(AH (l) W (l) ) (6)

[0080] In Equation (6), A is the normalized adjacency matrix, H(l + 1) is the feature representation of the (l + 1)-th layer, and W (l) is the trainable weight matrix of the l-th graph convolution layer, which is used to perform a linear transformation on the node feature matrix H (l) of the current layer, adjust the feature dimension and weight distribution to adapt to the model's need to capture features at different levels. W (l) is also combined with the normalized adjacency matrix A, and aggregates the feature information of neighboring nodes through matrix multiplication (AH (l) ), and then performs a parametric mapping on the aggregated features through W (l) to form the next layer of features H (l+1) . As a training parameter, W (l) is continuously optimized during the backpropagation process, enabling the model to more accurately extract the structural features (such as centrality and connectivity) of the fixation network, ultimately improving the prediction accuracy of the learning outcome.

[0081] In this alternative implementation, after inputting the structural variable into multiple graph convolution layers, the embedding representation H (K) of each node is obtained, and the global pooling layer (such as average pooling or max pooling) is used to aggregate the node embedding representations into a graph-level embedding representation h G , as shown in Equation (7):

[0082] h G = Pooling(H (K) ) (7)

[0083] Use a fully connected layer (FC) as shown in Equation (8) to map the graph-level embedding to the predicted value of the core result variable

[0084]

[0085] For the predicted value and the true value y, calculate the loss, such as mean squared error (MSE) or cross-entropy loss, and select an appropriate loss function according to the specific task. Specifically, a loss function as shown in Equation (9) can be adopted:

[0086]

[0087] Optimize the loss function to train the model parameters. Consistent with expectations, there is a relationship between the learning experience of the learning object in the IVR classroom and the fixation centrality of the learning object and the fixation connectivity among peers. It is worth noting that the structural features of the fixation-based fixation network can capture specific aspects of the visual attention distribution of the learning object, such as the centrality of fixating on a specific object of interest or different subgroups of visual attention (e.g., the proportion of boys in the observed group).

[0088] The method for determining the learning result provided by this alternative implementation determines that the learning result is only related to the features (e.g., centrality of content) that describe the visual attention related to different objects of interest in the structure of the fixation-based attention network of the learning object, while the more general description of the fixation behavior of the learning object (i.e., the sign of uniformity) is not related to any educational outcome examined.

[0089] In some alternative implementations of the present disclosure, for the above-mentioned fixation network, determining the structural variables includes: calculating the fixation centrality of a specific node in the fixation network; calculating the fixation connectivity between different nodes in the fixation network; calculating the uniformity of the fixation distribution of the learning object for each node in the fixation network; and using the fixation centrality, fixation connectivity, and fixation distribution uniformity as structural variables.

[0090] In this alternative implementation, the fixation centrality is used to characterize the importance of the virtual agent in the attention network of the learning object. For example, the fixation centrality includes degree centrality, and degree centrality (DC) measures the central degree of an object in fixation transfer.

[0091] In a possible implementation, the fixation centrality is evaluated through three variables regarding the center of interest (hereinafter referred to as OOIs) of the IVR classroom: (a) the centrality of the virtual learning object, (b) the centrality of the virtual teacher, and (c) the centrality of the content presenting the teaching content. Centrality, as an indicator for measuring the fixation centrality of OOIs, indicates the degree to which these OOIs are at the center of fixation transitions and describes the focus of attention on these OOIs. For each node in the graph, the centrality is defined as the sum of the weights of all incoming and outgoing edges (or the sum of the weights of all edges for more than one node). To calculate the fixation centrality, the frequencies of fixation transitions emanating from and pointing to the selected OOIs need to be determined. The frequencies of fixation transitions emanating from and pointing to the selected OOIs are determined through the following steps Step1 to Step3:

[0092] Step1: Capture and map data

[0093] The fixation point coordinates (x, y, z) and the line of sight direction of the learning object are captured in real time through the eye tracking module of the IVR device. Combining with the spatial coordinate system of the virtual environment, the fixation points are mapped onto specific OOIs. For example, when the line of sight of the learning object falls on the facial area of the virtual teacher, it is classified as a fixation on the "teacher" OOI; if the line of sight shifts to the raised hand action of a virtual peer in the front row, it is classified as a fixation on the "peer" OOI.

[0094] Step2: Count fixation transitions

[0095] The process of each fixation transition from one OOI to another is recorded. For example, when the learning object transfers from "teacher" to "content" and then to "peer", this sequence will be parsed into two edges: teacher → content, content → peer. The weight of each edge is determined by counting the number of fixation transitions of the learning object in this direction during the experiment. For example, if the "teacher → peer" transition occurs 50 times and the "peer → teacher" occurs 30 times, the edge weights in these two directions are 50 and 30 respectively.

[0096] Step3: Calculate fixation centrality

[0097] The calculation of fixation centrality (such as degree centrality) is based on a directed weighted graph. The centrality value of each OOI node is the sum of the weights of its incoming edges (the transition frequency from other OOIs to this node) and the sum of the weights of its outgoing edges (the transition frequency from this node to other OOIs). For example: if the "peer" node has incoming edge weights (teacher → peer = 50, content → peer = 40) and outgoing edge weights (peer → teacher = 30, peer → content = 60), then its centrality is 50 + 40 + 30 + 60 = 180. This calculation method reflects the pivotal role of this OOI in the overall fixation transition: the higher the centrality, the more frequently the learner's gaze transfers around this OOI.

[0098] In this alternative implementation, fixation connectivity is used to describe the fixation transfer relationship between different virtual agents, reflecting the attention switching of the learner among different virtual agents. Fixation connectivity includes: three variables regarding so-called cliques in the fixation network. A clique is a highly connected cluster (i.e., substructure) in the graph, thus providing information about fixation connectivity - that is, the scope of fixation transitions between OOIs in the fixation-based attention network. Since cliques can only be calculated in an undirected graph, each directed graph is converted into an undirected graph by calculating the weight of each undirected edge as the sum of the weights of two directed edges. In addition, the present disclosure also calculates all the maximum cliques (i.e., subsets containing the maximum number of nodes that have edges with each node in other subsets) among virtual learning objects. Since two connected nodes form a trivial clique, the present disclosure only considers cliques containing more than two nodes.

[0099] The process of converting a directed graph into an undirected graph mainly involves the following steps Step1’ to Step3’:

[0100] Step Step1’ traverses the edges in the directed graph. For each edge in the original directed graph (for example, the edge from node A to node B with weight W_AB), check whether there is a reverse edge (i.e., the edge from node B to node A with weight W_BA). If it exists, add the weights of the two edges (W_AB + W_BA) as the edge weight between node A and node B in the undirected graph. If there is only a one-way edge (for example, only A → B), then directly use the weight of this edge as the undirected edge weight.

[0101] Step Step2' Merge edges and construct an undirected graph. After summing the weights of all bidirectional edges, create an undirected edge connecting the corresponding nodes and assign the summed weight to this edge. For example: If the weight of the directed edge A→B is 5 and the weight of B→A is 3, then the weight of the undirected edge A - B is 5 + 3 = 8. If there is only a directed edge A→B with a weight of 5 and no edge B→A, then the weight of the undirected edge A - B is 5. This step ensures that each edge in the undirected graph retains only one instance, and the weight comprehensively reflects the total intensity of the two-way interaction.

[0102] Step Step3' Process weight symmetry. Since the edges of the undirected graph are essentially directionless, the above method retains the complete information of the two-way interaction in the original directed graph through summation. For example, a learning object frequently gazes from the "teacher" to the "peer" and then back from the "peer" to the "teacher" in an IVR classroom. This two-way attention is merged into an undirected edge with a high weight, reflecting the strong visual association between the two.

[0103] In this alternative implementation, the gaze distribution uniformity is used to measure whether the attention of the learning object is evenly distributed among different virtual entities. For example, whether it pays more attention to a specific object, or whether the attention is evenly distributed among multiple objects. The gaze distribution uniformity includes: weighted centrality measure, cut set size. The weighted centrality measure includes the uniformity of edge weights, and uses the weighted centrality of the content as an indicator of the even distribution of gaze from the content to different virtual learning objects. Secondly, the cut set size between the virtual entity and the virtual learning object is used as an indicator of the learning object's visual attention shifting back and forth between the two and staying on a certain group (for example, the learning object mainly focuses on the teacher / content). The cut set size is calculated by summing the edge weights of the edges transmitted between two subsets (one subset is the teacher and the content, and the other subset is all virtual learning objects). Thirdly, the present disclosure method examines the overall distribution of all edge weights in the network and tests (c) overall uniformity.

[0104] In this alternative implementation, the above-mentioned weighted centrality measure is an important indicator used to evaluate the attention distribution of a learning object in a virtual reality learning environment. This indicator is calculated based on the gaze transition frequency of the learning object between different objects of interest (such as virtual peers, virtual teachers, and teaching content), and the calculation process of the gaze transition frequency is as follows:

[0105] Record the fixation point data of the learning object during the experiment. These data include the coordinates and timestamps of the fixation points. Then, map these fixation points to specific objects in the virtual environment to determine which objects the learning object is looking at. Construct a fixation transition matrix that records the number of times the learning object transitions from one object to another. For example, if a learning object first fixates on a virtual teacher and then on a virtual peer, this transition will be recorded in the matrix. For each object, calculate the sum of the corresponding rows and columns in the matrix. This sum represents the importance of the object in the fixation transitions of the learning object. The higher the sum, the more important the object is in the attention distribution of the learning object. Normalize these sums to obtain a value between 0 and 1, which is the weighted centrality measure.

[0106] The method for determining structural variables provided by this optional implementation manner. The fixation network provides a framework for analyzing attention, and the structural variables are specific indicators in this framework. By quantifying and analyzing the attention distribution of the learning object through structural variables, the learning experience of the learning object is improved.

[0107] As Figure 3 shown, the immersive virtual reality interaction device 300 provided in this embodiment includes: a creation unit 301, a collection unit 302, a construction unit 303, and a determination unit 304. Among them, the above creation unit 301 can be configured to create an immersive virtual reality learning environment and configure the environmental attribute values of multiple virtual subjects in the immersive virtual reality learning environment. The above collection unit 302 can be configured to collect the fixation data of the learning object under the environmental attribute values. The above construction unit 303 can be configured to construct a fixation network based on the fixation data. The above determination unit 304 can be configured to determine the learning result of the learning object based on the fixation network.

[0108] In this embodiment, in the immersive virtual reality interaction device 300: the specific processing of the creation unit 301, the collection unit 302, the construction unit 303, and the determination unit 304 and the technical effects brought by them can be respectively referred to Figure 1 the relevant descriptions of steps 101, 102, 103, and 104 in the corresponding embodiments, which will not be elaborated here.

[0109] In some embodiments of the present disclosure, the above device further includes an obtaining unit (not shown in the figure), and the obtaining unit is configured to: optimize the environmental attribute value based on the learning result to obtain an optimized attribute value.

[0110] In some embodiments of the present disclosure, the obtaining unit of the above-mentioned apparatus is further configured to: determine the interaction degree and participation degree of the learning object based on the learning result; in response to the interaction degree and participation degree not meeting the interaction participation condition, adjust any one of the spatial position value, the visualization style value, and the action performance value to obtain an optimized attribute value.

[0111] In some embodiments of the present disclosure, the above-mentioned creating unit 301 is further configured to: create an immersive virtual reality learning environment including at least two virtual entities, where the virtual entities include: a virtual teacher and at least one virtual student; configure corresponding spatial position values for the positions between any two virtual entities; configure corresponding visualization style values and action performance values for each virtual entity.

[0112] In some embodiments of the present disclosure, the above-mentioned constructing unit 303 is further configured to: preprocess the gaze data to obtain preprocessed data; cluster the preprocessed data to determine the attention data of the learning object; obtain a gaze network based on the attention data.

[0113] In some embodiments of the present disclosure, the above-mentioned apparatus further includes a gaze network (not shown in the figure), and the gaze network is configured to: construct an initial attention network of the learning object, where nodes in the attention network represent learning objects, and the weights of the edges represent the attention relationships between learning objects; traverse the attention data of the learning object and add corresponding weights in the initial attention network; use a graph theory analysis algorithm to evaluate the influence and status of each learning object in the initial attention network to obtain the gaze network.

[0114] In some embodiments of the present disclosure, the above-mentioned determining unit 304 is further configured to: determine a structural variable based on the gaze network; input the structural variable into a pre-trained graph neural network model to obtain a learning result variable output by the graph neural network model; determine the learning result of the learning object based on the learning result variable.

[0115] In some embodiments of the present disclosure, the above-mentioned apparatus further includes a structural variable unit (not shown in the figure), and the structural variable unit is configured to: calculate the gaze centrality of a specific node in the gaze network; calculate the gaze connectivity between different nodes in the gaze network; calculate the gaze distribution uniformity of the learning object for each node in the gaze network; use the gaze centrality, the gaze connectivity, and the gaze distribution uniformity as structural variables.

[0116] The immersive virtual reality interaction device provided by the embodiments of the present disclosure, first, a creation unit 301 creates an immersive virtual reality learning environment and configures the environmental attribute values of multiple virtual entities in the immersive virtual reality learning environment; secondly, a collection unit 302 collects the gaze data of the learning object under the environmental attribute values; then, a construction unit 303 constructs a gaze network based on the gaze data; and finally, a determination unit 304 determines the learning result of the learning object based on the gaze network.

[0117] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0118] Figure 4 A schematic block diagram of an example electronic device 400 that can be used to implement the embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their modes are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0119] As Figure 4 shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0120] Multiple components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disc, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0121] The computing unit 401 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 executes the various methods and processes described above, such as the immersive virtual reality interaction method. For example, in some embodiments, the immersive virtual reality interaction method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the immersive virtual reality interaction method described above can be executed. Alternatively, in other embodiments, the computing unit 401 can be configured to execute the immersive virtual reality interaction method by any other suitable means (e.g., by means of firmware).

[0122] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), systems-on-a-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor, receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable immersive virtual reality interaction devices, such that when the program codes are executed by the processor or controller, the patterns / operations specified in the flowcharts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0124] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic, speech, or tactile input).

[0126] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0127] It should be understood that the various forms of the processes shown above can be used, steps can be reordered, added, or removed. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this is not limited herein.

[0128] The foregoing description of the specific exemplary embodiments of the present disclosure is for purposes of illustration and exemplification. These descriptions are not intended to limit the present disclosure to the precise forms disclosed, and it is apparent that many changes and variations are possible in light of the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present disclosure and its practical applications, so that those skilled in the art can implement and utilize the various different exemplary embodiments of the present disclosure, as well as various different selections and changes. The scope of the present disclosure is intended to be defined by the claims and their equivalents.

Claims

1. An immersive virtual reality interaction method, the method comprising: Creating an immersive virtual reality learning environment, and configuring environmental attribute values ​​of multiple virtual subjects in the immersive virtual reality learning environment; collecting gaze data of the learning object under the environment attribute value; Based on the gaze data, construct a gaze network; Based on the gaze network, a learning result of the learning object is determined.

2. The method according to claim 1, wherein the environmental attribute value comprises: Spatial position value, visualization style value, and action performance value; The method further comprises: Based on the learning results, determining the interaction and participation of the learning object; In response to the interactivity and the participation not satisfying an interactive participation condition, any one of the spatial position value, the visualization style value, and the action performance value is adjusted to obtain an optimized attribute value.

3. The method according to claim 1, wherein: The environmental attribute values ​​include: a spatial position value of a virtual subject, a visualization style value of a virtual subject, and an action performance value of the virtual subject. The step of creating an immersive virtual reality learning environment and configuring the environmental attribute values ​​of multiple virtual subjects in the immersive virtual reality learning environment includes: Creating an immersive virtual reality learning environment including at least two virtual subjects, the virtual subjects including: a virtual teacher and at least one virtual student; Configure corresponding spatial position values ​​for the positions between any two virtual subjects; Configure corresponding visualization style values ​​and action performance values ​​for each virtual subject.

4. The method according to claim 1, wherein: The constructing the gaze network based on the gaze data comprises: Preprocessing the gaze data to obtain preprocessed data; Clustering the preprocessed data to determine the focus data of the learning object; Based on the attention data, a gaze network is obtained.

5. The method according to claim 4, wherein: The obtaining of the gaze network based on the attention data comprises: Constructing an initial attention network of the learning object, in which nodes represent learning objects and edge weights represent attention relationships between learning objects; Traversing the attention data of the learning object and adding corresponding weights to the initial attention network; A graph analysis algorithm is used to evaluate the influence and status of each learning object in the initial attention network to obtain a gaze network.

6. The method according to claim 1, wherein: The determining, based on the gaze network, a learning result of the learning object comprises: Based on the gaze network, determining structural variables; Inputting the structural variables into a pre-trained graph neural network model to obtain learning result variable values ​​output by the graph neural network model; Based on the learning result variable value, a learning result of the learning object is determined.

7. The method according to claim 6, wherein: Determining the structural variables based on the gaze network comprises: calculating the gaze centrality of a specific node in the gaze network; Calculating gaze connectivity between different nodes in the gaze network; Calculating the uniformity of the gaze distribution of the learning object on each node in the gaze network; The fixation centrality, the fixation connectivity and the fixation distribution uniformity are taken as structural variables.

8. An immersive virtual reality interactive device, comprising: A creation unit configured to create an immersive virtual reality learning environment and configure environmental attribute values ​​of multiple virtual subjects in the immersive virtual reality learning environment; A collection unit configured to collect gaze data of the learning object under the environment attribute value; A construction unit, configured to construct a gaze network based on the gaze data; A determination unit is configured to determine a learning result of the learning object based on the gaze network.

9. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 7.

Images