Mixed reality interaction method and device based on human factor calculation, equipment and storage medium

Abstract

The invention discloses a mixed reality interaction method and device based on human factor calculation, equipment and a storage medium, and relates to the technical field of mixed reality, and the method comprises the steps: determining the space color information of a virtual reality space according to the emotion information of a user; determining the space regularity and the space opening degree of the virtual reality space based on the human skeleton information of the user; determining a space enclosing form of the virtual reality space based on the head position information of the user; constructing a Rhino grid model based on the space color information, the space regularity, the space opening degree and the space enclosing form; and displaying the virtual reality space in the head display device through the Rhino grid model so as to perform real-time interaction with the user. By applying the technical scheme, the technical problem that intelligent services cannot be provided for users due to the fact that subconsciousness requirements of the users are difficult to capture and mine in a mixed reality man-machine interaction mode in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of mixed reality technology, and in particular to a mixed reality interaction method, device, equipment and storage medium based on human factors computing. Background Art

[0002] Mixed reality (MR) is a new visual environment created by merging the real world and the virtual world. It enables the coexistence of physical and digital objects and real-time interaction. With the development of computer hardware and software technology, mixed reality technology is increasingly used in life and work, bringing the possibility of intelligent interaction between many people and spaces. However, in the process of human-computer interaction, user needs are often generated without the user's awareness and cannot be actively input into the machine end of the virtual environment. Existing mixed reality technology lacks effective methods to detect and respond to these implicit needs, and thus cannot provide more intelligent services. Summary of the invention

[0003] The main purpose of this application is to provide a mixed reality interaction method, device, equipment and storage medium based on human factors computing, aiming to solve the technical problem that the mixed reality human-computer interaction method in the prior art is difficult to capture and tap the user's subconscious needs, resulting in the inability to provide intelligent services to users.

[0004] To achieve the above objectives, the present application proposes a mixed reality interaction method based on human factors computing, the method comprising:

[0005] Determine spatial color information of the virtual reality space according to the user's emotional information;

[0006] Determining the spatial regularity and spatial openness of the virtual reality space based on the human skeleton information of the user;

[0007] Determining a spatial enclosure form of the virtual reality space based on the head position information of the user;

[0008] Constructing a Rhino grid model based on the spatial color information, the spatial regularity, the spatial openness and the spatial enclosure form;

[0009] The virtual reality space is displayed in a head display device through the Rhino grid model to interact with the user in real time.

[0010] In one embodiment, the step of determining the spatial color information of the virtual reality space according to the user's emotional information includes:

[0011] Collecting the user's facial expressions, and obtaining the user's emotional information based on the facial expressions through an emotion recognition algorithm, wherein the emotional information includes several emotional dimensions;

[0012] Quantifying each emotion dimension to obtain an emotion value corresponding to the emotion dimension;

[0013] The spatial color information of the virtual reality space is determined based on the emotion value.

[0014] In one embodiment, the step of determining the spatial color information of the virtual reality space based on the emotion value comprises:

[0015] Determine a target emotion dimension from the emotion dimensions based on the emotion value, the target emotion dimension being the emotion dimension with the largest emotion value among the emotion dimensions;

[0016] Determining the spatial color saturation of the virtual reality space according to the target emotion value corresponding to the target emotion dimension;

[0017] Determining the spatial hue of the virtual reality space according to the matching color corresponding to the target emotion value;

[0018] Spatial color information of the virtual reality space is determined based on the spatial color saturation and the spatial hue.

[0019] In one embodiment, the step of determining the spatial regularity and spatial openness of the virtual reality space based on the human skeleton information of the user comprises:

[0020] Determine a first angle, a second angle, a third angle and a fourth angle corresponding to the user based on the human skeleton information of the user, wherein the first angle is the angle between the spine and the thigh of the user, the second angle is the angle between the thigh and the calf of the user, the third angle is the angle between the upper arm and the forearm of the user, and the fourth angle is the angle between the upper arm and the upper body of the user;

[0021] determining the user's sitting posture flexibility according to the first angle and the second angle;

[0022] Determining the thinking intensity of the user according to the third angle and the fourth angle;

[0023] Determining the spatial openness of the virtual reality space based on the sitting posture flexibility of the user;

[0024] The spatial regularity of the virtual reality space is determined based on the thinking intensity of the user.

[0025] In one embodiment, the step of determining the spatial enclosure form of the virtual reality space based on the head position information of the user includes:

[0026] Determine whether a target user exists within a preset range of the user, and obtain a first determination result;

[0027] If the first judgment result is yes, judging whether the user and the target user are on the same side based on a preset separation position, and obtaining a second judgment result;

[0028] Determining a first head orientation corresponding to the user based on the head position information of the user;

[0029] determining a communication mode between the user and the target user based on the first head orientation and a second head orientation corresponding to the target user;

[0030] The spatial enclosure form of the virtual reality space is determined based on the first judgment result, the second judgment result and the interaction mode.

[0031] In one embodiment, the step of determining the interaction mode between the user and the target user based on the first head orientation and a second head orientation corresponding to the target user includes:

[0032] Acquire a first connection angle between the first head orientation and a head line, and a second connection angle between a second head orientation corresponding to the target user and the head line, the head line being a line between the heads of the user and the target user;

[0033] The interaction mode between the user and the target user is determined based on the first connection angle and the second connection angle.

[0034] In one embodiment, after the step of determining the spatial enclosure form of the virtual reality space based on the first judgment result, the second judgment result and the interaction mode, the method further includes:

[0035] When the virtual reality space is in a spatially enclosed form, determining a front and rear head orientation angle of the user within a target time interval based on the head position information;

[0036] Determining a frequency of a change in the user's head orientation according to the front-to-back head orientation angle and a preset head angle threshold;

[0037] Determining the attention state corresponding to the user according to the frequency of the head orientation change;

[0038] determining a spatially opposite height of the virtual reality space based on the attention state;

[0039] The step of constructing a Rhino grid model based on the spatial color information, the spatial regularity, the spatial openness and the spatial enclosure form includes:

[0040] A Rhino grid model is constructed based on the color information of the space, the regularity of the space, the openness of the space and the height of the opposite side of the space.

[0041] In addition, to achieve the above objectives, the present application also proposes a mixed reality interaction device based on human factors computing, the device comprising:

[0042] A color information determination module, used to determine the spatial color information of the virtual reality space according to the user's emotional information;

[0043] A spatial information determination module, used to determine the spatial regularity and spatial openness of the virtual reality space based on the human skeleton information of the user;

[0044] The spatial information determination module is further used to determine the spatial enclosure form of the virtual reality space based on the head position information of the user;

[0045] A model building module, used for building a Rhino grid model based on the spatial color information, the spatial regularity, the spatial openness and the spatial enclosure form;

[0046] A human-computer interaction module is used to display the virtual reality space in a head display device through the Rhino grid model so as to interact with the user in real time.

[0047] In addition, to achieve the above-mentioned purpose, the present application also proposes a mixed reality interaction device based on human factors computing, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the mixed reality interaction method based on human factors computing as described above.

[0048] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the mixed reality interaction method based on human factors computing as described above are implemented.

[0049] The present application provides a mixed reality interaction method based on human factors computing, and the present application discloses determining the spatial color information of a virtual reality space according to the user's emotional information; determining the spatial regularity and spatial openness of the virtual reality space based on the user's human skeleton information; determining the spatial enclosure form of the virtual reality space based on the user's head position information; constructing a Rhino grid model based on the spatial color information, spatial regularity, spatial openness and spatial enclosure form; and displaying the virtual reality space in a head display device through the Rhino grid model to interact with the user in real time. The present invention determines the spatial color information, spatial regularity, spatial openness and spatial enclosure form of the virtual space based on the user's information, and displays the virtual space through the constructed Rhino grid model to interact with the user in real time, thereby solving the technical problem that the mixed reality human-computer interaction method in the prior art is difficult to capture and tap the user's subconscious needs, resulting in the inability to provide intelligent services to the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0051] Figure 1 A flowchart diagram of the first embodiment of the mixed reality interaction method based on human factors computing provided in this application;

[0052] Figure 2 A flowchart diagram of Embodiment 2 of the mixed reality interaction method based on human factors computing provided in this application;

[0053] Figure 3 A flowchart of determining the emotion value corresponding to the emotion dimension in the mixed reality interaction method based on human factor computing in this application;

[0054] Figure 4 A schematic diagram of the change of spatial color in the mixed reality interaction method based on human factors computing in this application;

[0055] Figure 5 A schematic diagram of determining the user's sitting flexibility and thinking intensity in the mixed reality interaction method based on human factors computing in this application;

[0056] Figure 6 A schematic diagram of changes in spatial openness and spatial regularity in the mixed reality interaction method based on human factor computing in this application;

[0057] Figure 7 A flowchart diagram of Embodiment 3 of the mixed reality interaction method based on human factors computing provided in this application;

[0058] Figure 8A schematic diagram of determining the user's interaction mode in the mixed reality interaction method based on human factors computing in this application;

[0059] Fig. 9 A schematic diagram of the change of the spatial enclosure form in the mixed reality interaction method based on human factors computing in this application;

[0060] Fig.10 A flowchart for determining the user's attention state in the mixed reality interaction method based on human factors computing in this application;

[0061] Fig.11 A schematic diagram of the change of the height of the opposite side of the space in the mixed reality interaction method based on human factor computing in this application;

[0062] Fig.12 A schematic diagram of the process of user behavior driving spatial changes in the mixed reality interaction method based on human factors computing in this application;

[0063] Fig.13 This is a schematic diagram of the overall process of the mixed reality interaction method based on human factors computing in this application;

[0064] Fig.14 This is a schematic diagram of the module structure of a mixed reality interaction device based on human factors computing in an embodiment of the present application;

[0065] Fig.15 Schematic diagram of the device structure of the hardware operating environment involved in the mixed reality interaction method based on human factors computing in the embodiment of the present application. DETAILED DESCRIPTION

[0066] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0067] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0068] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a mixed reality interactive device, etc. The following takes the mixed reality interactive device as an example (hereinafter referred to as the device) to illustrate this embodiment and the following embodiments.

[0069] Based on this, the embodiment of the present application provides a mixed reality interaction method based on human factor computing, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the mixed reality interaction method based on human factors computing of the present application.

[0070] In this embodiment, the mixed reality interaction method based on human factors computing includes steps S10 to S50:

[0071] Step S10: Determine the spatial color information of the virtual reality space according to the user's emotional information.

[0072] It should be understood that the user's emotional information may be information used to reflect the current user's emotional state. In this embodiment, the device may capture the user's facial expression through a depth camera, and then identify the user's emotional information in four dimensions of happiness, sadness, anger, and surprise through an emotion recognition algorithm.

[0073] It should be noted that the above virtual reality space is a virtual reality scene. In practical applications, the mixed reality interaction method based on human factors computing proposed in this scheme may include three parts: perception-computation-effect, wherein the hardware of the perception part is completed by a depth camera and a VR headset (virtual reality head-mounted display device); the calculation part uses Grasshopper battery processing operations; the effect end uses Meta Quest 3 as a medium for presenting the virtual reality scene, and the overall feedback loop is completely automatically completed by the control scheme, realizing the real-time response and enhancement of the space to the human behavior state. Among them, the Grasshopper battery is an important component of the Rhino 3D modeling software. It provides a series of functional components to enhance the functions of Rhino and help users complete complex modeling tasks more efficiently. In this embodiment, the effect part is first implemented in Rhino Grasshopper through parametric modeling, and then Meta quest 3 is used to present the final virtual scene. In Rhino, a series of points can be used as vertices to form multiple surfaces to form a space, and then the surface UV is divided into equal parts and converted into a mesh surface, thereby constructing the above virtual reality space.

[0074] It can be understood that the above-mentioned spatial color information is information used to characterize the color of the virtual reality space. The spatial color information in this embodiment may include but is not limited to spatial color saturation, spatial hue, etc.

[0075] Step S20: determining the spatial regularity and spatial openness of the virtual reality space based on the human skeleton information of the user.

[0076] It should be understood that the above-mentioned human skeleton information can be the coordinate information of each bone point in the user's skeleton, for example: the user's left and right shoulder coordinates, left and right elbow coordinates, left and right wrist coordinates, etc., and this embodiment does not limit this.

[0077] It should be noted that the above-mentioned spatial regularity can be a value used to characterize the regularity of the form of the virtual reality space; the above-mentioned spatial openness can be a value used to characterize the openness of the virtual reality space. In practical applications, the spatial regularity of the virtual reality space is affected by the user's thinking intensity, and the strength of the user's thinking intensity is determined based on the angle between the user's upper arm and forearm and the angle between the upper arm and the upper body. The stronger the user's thinking intensity, the more regular the spatial form of the virtual reality space, which is manifested as a flat enclosed surface; the lower the thinking intensity, the more complex the spatial form, which is manifested as an uneven enclosed surface. The spatial openness of the virtual reality space is affected by the flexibility of the user's overall sitting posture, and the flexibility of the user's sitting posture is determined by the angle between the user's spine and thigh and the angle between the thigh and calf. The higher the user's sitting posture flexibility, the more open the virtual reality space is, which is manifested as a random reduction in the grid surface constituting the space; the lower the sitting posture flexibility, that is, the more systematic the sitting posture, the more closed the space, and the more complete the grid surface. Therefore, in this embodiment, the user's thinking intensity and sitting flexibility can be first determined based on the user's human skeleton information, and then the spatial regularity and spatial openness of the virtual reality space can be determined respectively based on the user's thinking intensity and sitting flexibility.

[0078] Step S30: Determine the spatial enclosure shape of the virtual reality space based on the head position information of the user.

[0079] It can be understood that the user's head position information may be information used to characterize the orientation of the user's head. In addition, the user's head position information in this embodiment may also include coordinate point information of the user's head.

[0080] It should be noted that the spatial enclosure form of the virtual reality space is the separation state of the virtual reality space. In this embodiment, the composition logic of the virtual reality space is to form multiple curved surfaces with a series of points as vertices, and the curved surfaces are combined together to form a space, and then the curved surface UV is equally divided and converted into a mesh surface, wherein the four vertices above the plane area dividing line can move downward along the specified curve direction, thereby changing the separation state of the space. In this embodiment, it can be determined whether the separation conditions of the virtual reality space are met based on the user's head position information. If it is met, the vertex gradually moves downward along the specified straight line direction to 0.3 meters from the ground, and the space is separated into two parts from the middle; if it is not met, the vertex moves upward along the straight line to make the space connected.

[0081] It should be noted that the above-mentioned opposite height of the space is the height of the opposite point of the space in the virtual reality space. In practical applications, the opposite height of the space is controlled by the attention state. The attention state is quantified as a value in the range of 0 to 1 in the behavior calculation. The higher the attention state value, the lower the opposite space height. In this embodiment, the spatial area that needs to change can be first determined based on the user's position. Among them, there are 18 vertices controlled on the top surface of the space. In this embodiment, the distance between the human head coordinate point and the top surface vertex can be calculated and sorted, and 8 points with larger distances can be selected as the opposite points of the space in the virtual reality space. Then, the user's attention state can be determined based on the user's head state information, and the height of the opposite point of the space in the virtual reality space can be determined based on the user's attention state.

[0082] Step S40: constructing a Rhino grid model based on the spatial color information, the spatial regularity, the spatial openness and the spatial enclosure form.

[0083] It should be understood that the above Rhino mesh model can be a mesh model established in Rhino for representing the physical form of the virtual reality space. Rhino is a 3D modeling software. In addition to Rhino software, other 3D modeling software with the same or similar functions as Rhino software can also be used in this solution, and this embodiment does not limit this.

[0084] In this embodiment, the device can calculate the spatial color information, spatial regularity, spatial openness, spatial enclosure form and height of the opposite side of the virtual reality space through the Grasshopper cell in the preset modeling software (such as Rhino), and standardize these data into formatted data from 0 to 1, wherein a part of the 0 to 1 data is used to control the position of the midpoint of the space, thereby changing the enclosure form, height, and regularity of the space; the other part of 0 to 1 is used for mesh surface adjustment, including defining the mesh vertex color and changing the number of mesh surfaces, thereby changing the color and openness of the space, and finally constructing a Rhino mesh model.

[0085] In practical applications, this solution can be verified for feasibility with the help of the smart office space SENSE BOX as a prototype. SENSE BOX mainly focuses on four types of working states of people, covering four dimensions: task processing mode, attention state, communication mode, and emotional state. Among them, the task processing mode part uses a depth camera to capture the human skeletal joints, and the flexibility is judged by the angle of the human torso, and the thinking intensity is judged by the upper limb joints. The more flexible the sitting posture, the higher the openness of the virtual space, providing more possibilities for communication and relaxation with the outside world; the higher the thinking intensity, the higher the regularity of the space, so that people can focus more on thinking about the tasks that need to be processed. The attention state part uses a VR headset to capture the head orientation, and the frequency of head orientation changes over a period of time is used to judge the degree of concentration. The more concentrated the attention, the lower the height of the opposite side of the space, reducing the interference of other people or things in the space on their own attention. The communication mode part uses a depth camera to capture the head orientation and relative position relationship of multiple people in the space. When the angle between the head orientation of two people in the space and the line connecting the two people is less than a certain value range, it is judged that the two people are in a communication state. When they are in a communication state, the corresponding separation changes will be made according to the positions of the two people, thereby meeting the different work needs of people in the space to work independently and cooperate with each other. The emotional state part uses a depth camera to capture people's facial expressions, and the emotion recognition algorithm determines the four emotions of happiness, sadness, anger, and surprise. These four emotions correspond to four spatial atmosphere colors. The more obvious the emotion, the higher the saturation of the spatial color.

[0086] Step S50: displaying the virtual reality space in a head display device through the Rhino grid model to interact with the user in real time.

[0087] It should be noted that after obtaining the Rhino mesh model, the device can pass the Rhino mesh model to Unity, so that a real-scale virtual reality space can be presented in a head display device (such as a VR headset), thereby realizing real-time interaction between the user and the device.

[0088] It should be noted that the method proposed in this scheme can automatically perceive the state of people in the space and provide corresponding feedback in the mixed reality medium. In this way, the device can adjust the environment and interactive content in real time according to the actual needs and emotional changes of the user, thereby improving the user's comfort and overall experience in the mixed reality world. In addition, this intelligent feedback mechanism not only reduces the user's operating burden, but also makes human-computer interaction more natural and smooth.

[0089] In practical applications, the user's human skeleton information can be obtained in real time through the ZED depth camera, and the user's head position information can be obtained in real time through the VR headset (Meta Quest 3). The above two pieces of information are used as input. At the same time, the depth camera can also be used to obtain relevant images. The obtained information is first processed preliminarily through Python and C# respectively. After the preliminary processing, the data can be transmitted to Grasshopper through the UDP protocol for calculation and formatting. The processed spatial effect data is transmitted to Unity through Rhino Inside Unity, and the mixed reality scene is presented in the VR headset.

[0090] The present embodiment provides a mixed reality interaction method based on human factors computing. Since the present embodiment can determine the spatial color information, spatial regularity, spatial openness and spatial enclosure form of the virtual space based on the user's information, and display the virtual space through the constructed Rhino grid model to interact with the user in real time, it solves the technical problem that the mixed reality human-computer interaction method in the prior art is difficult to capture and tap the user's subconscious needs, resulting in the inability to provide intelligent services to users.

[0091] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , Figure 2 A flowchart diagram of Embodiment 2 of the mixed reality interaction method based on human factors computing provided in this application.

[0092] In this embodiment, step S10 includes steps S101 to S103:

[0093] Step S101: collecting the user's facial expressions, and obtaining the user's emotional information based on the facial expressions through an emotion recognition algorithm, wherein the emotional information includes several emotional dimensions.

[0094] It should be understood that the above-mentioned emotion recognition algorithm may be an algorithm for automatically identifying the emotional state of a user.

[0095] Step S102: quantify each emotion dimension to obtain an emotion value corresponding to the emotion dimension.

[0096] In this embodiment, refer to Figure 3 , Figure 3 This is a flow chart of determining the emotion value corresponding to the emotion dimension in the mixed reality interaction method based on human factor computing in this application. Figure 3As shown, the emotional dimensions in the user's emotional information may include: pleasure, sadness, anger and surprise. The device can capture the user's emotions through an emotion recognition algorithm, and recognize the user's pleasure, sadness, anger and surprise based on the user's facial expression. The emotion values ​​corresponding to the emotion dimensions in this embodiment can be divided into 5 levels (0, 0.25, 0.5, 0.75, 1), where 0 means very dissimilar; 0.25 means dissimilar; 0.5 means possible; 0.75 means similar; 1 means very similar. In addition, in a multi-person scenario, the device can be combined with a VR headset for emotion recognition, and can then distinguish the emotions of users wearing VR headsets.

[0097] Step S103: determining the spatial color information of the virtual reality space based on the emotion value.

[0098] Specifically, the step S103 includes:

[0099] Step S103a: determining a target emotion dimension from the emotion dimensions based on the emotion value, wherein the target emotion dimension is the emotion dimension having the largest emotion value among the emotion dimensions.

[0100] Step S103b: determining the spatial color saturation of the virtual reality space according to the target emotion value corresponding to the target emotion dimension.

[0101] It is understandable that the above-mentioned space color saturation may be a value used to characterize the vividness of the color of the virtual reality space. In this embodiment, the higher the target emotion value corresponding to the target emotion dimension, the higher the saturation of the grid color corresponding to the virtual reality space, that is, the higher the above-mentioned space color saturation.

[0102] Step S103c: determining the spatial hue of the virtual reality space according to the matching color corresponding to the target emotion value.

[0103] It should be noted that the spatial hue can be a feature used to describe the grid color of the virtual reality space. The spatial grid color of the virtual reality space is dominated by emotions. In this embodiment, the matching colors corresponding to the four emotional dimensions of joy, sadness, anger, and surprise can be set to red, gray, blue, and green, respectively. In the calculation, the four emotional dimensions can be quantified into four values ​​in the range of 0 to 1. In practical applications, when the value of each emotion is the same or no emotion is detected in the space, the spatial grid vertices are assigned white; when one emotion value is higher than the others, the grid vertices are assigned the color corresponding to this emotion, and the higher the value of this emotion, the higher the saturation of the grid color.

[0104] Step S103d: determining the spatial color information of the virtual reality space based on the spatial color saturation and the spatial hue.

[0105] In the specific implementation, refer to Figure 4 , Figure 4 This is a schematic diagram of the change of spatial color in the mixed reality interaction method based on human factors computing in this application. Figure 4 In this example, we take pleasure as the target emotion dimension (i.e., pleasure value accounts for the largest proportion of all emotion values). Figure 4 As shown in the figure, when the pleasure value is between 0 and 0.25, the color of the spatial grid is white (255, 255, 255); when the pleasure value is between 0.25 and 0.5, the color of the spatial grid is low-saturation red (240, 194, 194); when the pleasure value is between 0.5 and 0.75, the color of the spatial grid is high-saturation red (227, 107, 107); when the pleasure value is between 0.75 and 1, the color of the spatial grid is high-saturation red (235, 91, 91). The transition between different color states is smooth. For example, when the pleasure value changes from 0.2 to 0.6, the spatial grid gradually changes from white to high-saturation red, and the change time is about 1 second.

[0106] Furthermore, the step S20 includes:

[0107] Step S201: Determine the first angle, second angle, third angle and fourth angle corresponding to the user based on the user's human skeleton information, where the first angle is the angle between the user's spine and thigh, the second angle is the angle between the user's thigh and calf, the third angle is the angle between the user's upper arm and forearm, and the fourth angle is the angle between the user's upper arm and upper body.

[0108] Step S202: determining the user's sitting flexibility according to the first angle and the second angle.

[0109] It should be understood that the above user sitting posture flexibility can be a value used to characterize the flexibility of the user's sitting posture. Figure 5 , Figure 5 This is a schematic diagram of determining the user's sitting flexibility and thinking intensity in the mixed reality interaction method based on human factor computing in this application. Figure 5 As shown, in this embodiment, the difference between the angle between the user's spine and thigh and the angle between the thigh and calf and 90° can be used to determine whether the person's sitting posture is "flexible" or "systematic", wherein the closer the angle is to 90°, the more "systematic" the sitting posture is. The calculation formula is as follows:

[0110]

[0111] In the formula, α is the first angle, β is the second angle, α′ is the difference between the first angle and 90°, β′ is the difference between the second angle and 90°, and X is a value used to characterize the flexibility of the user's sitting posture.

[0112] Step S203: Determine the user's thinking intensity according to the third angle and the fourth angle.

[0113] In this embodiment, if Figure 5 As shown, in this embodiment, the angle between the upper arm and the forearm (elbow angle) and the angle between the upper arm and the upper body can be used to judge whether the thinking intensity of a person is strong or weak, wherein the smaller the angle, the higher the thinking intensity of the user, and the calculation formula is as follows:

[0114]

[0115] In the formula, α left is the angle between the user's left upper arm and left forearm (i.e., the third angle corresponding to the user's left side), α right is the angle between the user's right upper arm and right forearm (i.e., the third angle corresponding to the user's right side), β left is the angle between the user's left upper arm and upper body (i.e., the fourth angle corresponding to the left side), β right is the angle between the user's right upper arm and upper body (i.e., the fourth angle corresponding to the right side), α′ is the average of the angles between the user's left upper arm and left forearm and the angles between the right upper arm and the lower forearm, β′ is the average of the angles between the user's left upper arm and upper body and the angles between the right upper arm and upper body, and X is a value used to characterize the user's thinking intensity.

[0116] Step S204: determining the spatial openness of the virtual reality space based on the user's sitting posture flexibility.

[0117] It should be noted that the openness of the space is affected by the flexibility of the overall sitting posture of the person. In this embodiment, the sitting posture flexibility can be quantified as a value in the range of 0 to 1 in the behavior calculation. Among them, the higher the sitting posture flexibility, the fewer grid surfaces constituting the space are randomly reduced, and the space is more open; the lower the sitting posture flexibility, that is, the more systematic the sitting posture, the more complete the grid surface, and the more closed the space.

[0118] In practical applications, refer to Figure 6 , Figure 6 This is a schematic diagram of the changes in spatial openness and spatial regularity in the mixed reality interaction method based on human factor computing in this application. Figure 6As shown in the figure, if there are 230 grids in the space (each group consists of two grids on the front and back), when the sitting flexibility is within 0-0.25, the grids do not change; when the sitting flexibility is within 0.25-0.5, the grids are randomly reduced, reducing 0.15 of the total number of grids, that is, 35 grids are reduced, and the position of the reduction is random; when the sitting flexibility is within 0.5-0.75, the total number of grids is reduced by 0.35, that is, 81 grids are reduced; when the sitting flexibility is within 0.75-1, the total number of grids is reduced by 0.5, that is, 115 grids are reduced. Among them, the transformation between these four states is smooth. For example, when the sitting flexibility changes from 0.3 to 0.6, the number of reduced grids will gradually increase from 35 groups to 81 groups, and the entire change time is about 1 second.

[0119] Step S205: Determine the spatial regularity of the virtual reality space based on the user's thinking intensity.

[0120] It should be noted that the regularity of space is affected by the intensity of thinking. In this embodiment, the intensity of thinking can be quantified as a value in the range of 0 to 1 in the behavior calculation. The higher the intensity of thinking, the flatter the surface of the space enclosure, and the more regular the space form; the lower the intensity of thinking, the more uneven the surface of the space enclosure, and the more complex the space form.

[0121] In practical applications, such as Figure 6 As shown, when the thinking intensity is within 0-0.25, the vertex is at the end point of the curve on the outside, and the space is the most complex and irregular; when the thinking intensity is within 0.25-0.5, the vertex is at a position 1 / 3 of the curve length from the end point, and the space is relatively irregular; when the thinking intensity is within 0.5-0.75, the vertex is at a position 2 / 3 of the curve length from the end point, and the space is relatively regular; when the thinking intensity is within 0.75-1, the vertex is at the end point on the inside of the curve, and the space is the most regular. Among them, the transformation between the four states is smooth. For example, when the thinking intensity changes from 0.2 to 0.6, the vertex gradually moves from the outside end point position along the specified curve to the position 2 / 3 of the curve length from the end point, and the space gradually changes from the most irregular to the more regular, and the change time is about 1 second.

[0122] In this embodiment, it is disclosed that facial expressions of users are collected, and emotional information of users is obtained based on facial expressions through an emotion recognition algorithm, wherein the emotional information includes several emotional dimensions; each emotional dimension is quantified to obtain emotional values ​​corresponding to the emotional dimensions; spatial color information of a virtual reality space is determined based on the emotional values, so that human-computer interaction can be performed based on user emotions in the future, thereby improving the intelligence of human-computer interaction and further improving user experience.

[0123] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction, and will not be described in detail later. Figure 7 , Figure 7 A flowchart diagram of the third embodiment of the mixed reality interaction method based on human factors computing of the present application is provided.

[0124] In this embodiment, step S30 includes steps S301 to S306:

[0125] Step S301: determine whether a target user exists within a preset range of the user, and obtain a first determination result.

[0126] It can be understood that the above-mentioned preset range can be a certain range around the user, and this embodiment does not limit the specific size of the preset range; accordingly, the above-mentioned target user can be any user within the preset range of the user.

[0127] Step S302: If the first judgment result is existence, it is judged whether the user and the target user are on the same side based on a preset separation position to obtain a second judgment result.

[0128] In this embodiment, the device can determine whether the user and the target user are on the same side based on the preset separation position. If the user and the target user are both distributed on one side of the preset separation position, it means that the user and the target user are on the same side; if the user and the target user are respectively distributed on both sides of the preset separation position, it means that the user and the target user are not on the same side.

[0129] Step S303: Determine a first head orientation corresponding to the user based on the head position information of the user.

[0130] Step S304: determining a communication mode between the user and the target user based on the first head orientation and a second head orientation corresponding to the target user.

[0131] It should be understood that the first head orientation and the second head orientation are the orientations of the heads of the user and the target user respectively.

[0132] It should be noted that the above-mentioned communication mode can be a mode of getting along between the user and the target user. The communication mode in this embodiment may include: a communication state and a non-communication state, wherein the communication state is used to represent that there is communication between the user and the target user; the non-communication state represents that there is no communication between the user and the target user.

[0133] Specifically, the step S304 includes: obtaining a first connection angle between the first head orientation and the head line, and a second connection angle between the second head orientation corresponding to the target user and the head line, the head line being the line between the heads of the user and the target user; determining the interaction mode between the user and the target user based on the first connection angle and the second connection angle.

[0134] In this embodiment, it can be determined whether the user and the target user are communicating based on the angle between the head orientations of the user and the target user and the straight line connecting the two. Specifically, if the angle between the first head orientation of the user and the first connecting line of the head and the angle between the second head orientation of the target user and the second connecting line of the head are both within 45°, it is determined that the user and the target user are currently in a communication state, otherwise it is determined that the user and the target user are not currently in a communication state.

[0135] Step S305: determining the spatial enclosure form of the virtual reality space based on the first judgment result, the second judgment result and the interaction mode.

[0136] In this embodiment, refer to Figure 8 , Figure 8 This is a schematic diagram of determining the user's interaction mode in the mixed reality interaction method based on human factors computing in this application. First, it can be determined whether the user and the target user are in a state of interaction based on the direction of the head, such as Figure 8 As shown, when the angle α between the first head orientation corresponding to the user and the first line connecting the head is less than or equal to 45° and the angle β between the second head orientation corresponding to the target user and the second line connecting the head is equal to 45°, interaction = True, i.e. the user is in interaction mode. In addition, in this embodiment, the number of users within a certain range and the relative position of the fixed segmentation can be combined to determine whether the virtual reality space is divided effectively. Figure 8As shown, if according to the first judgment result, it is found that the number of people within the preset range around the user is less than or equal to 1 or greater than or equal to 3, the space is not separated; if according to the first judgment result, it is found that the number of people within the preset range around the user is 2, and according to the second judgment result, it is found that the target and the target user are on the same side, and the target and the target user are in a communication mode, then the space is not separated; if according to the first judgment result, it is found that the number of people within the preset range around the user is 2, and according to the second judgment result, it is found that the target and the target user are separated on both sides, and the target and the target user are in a communication mode, then the space is not separated; if according to the first judgment result, it is found that the number of people within the preset range around the user is 2, and according to the second judgment result, it is found that the target and the target user are on the same side, and the target and the target user are not in a communication mode, then the space is not separated; if according to the first judgment result, it is found that the number of people within the preset range around the user is 2, and according to the second judgment result, it is found that the target and the target user are separated on both sides, and the target and the target user are not in a communication mode, then the space is separated.

[0137] In practical applications, refer to Fig. 9 , Fig. 9 This is a schematic diagram of the change of the spatial enclosure form in the mixed reality interaction method based on human factor computing in this application. Fig. 9 As shown, in this embodiment, the user's communication mode can cause the space enclosure form to change, wherein the space is separated when and only when the following conditions are met: the number of people in the range = 2, two people are separated on both sides (separation state = False), and two people are not in a communication state (communication state = False), so as to meet the user's communication and solitude needs. When the number of people in the range is equal to 1, the number of people in the range = 2, the separation state = False, and the communication state = False, or when the number of people in the range = 2, the separation state = True, and the communication state = True, the space is not separated.

[0138] Furthermore, after step S305, the method further includes: when the virtual reality space is in a spatial enclosure form, determining a front-to-back head orientation angle of the user within a target time interval based on the head position information; determining a frequency of head orientation change of the user according to the front-to-back head orientation angle and a preset head angle threshold; determining an attention state corresponding to the user according to the head orientation change frequency; and determining a spatial opposite side height of the virtual reality space based on the attention state.

[0139] Accordingly, the step S40 includes: constructing a Rhino grid model based on the spatial color information, the spatial regularity, the spatial openness and the height of the opposite side of the space.

[0140] It should be noted that the front-to-back head orientation angle may be the angle of the user's head orientation before and after the target time interval. For example, if the target time interval is 1 second, the front-to-back head orientation angle is the angle of the user's head orientation at the current moment and one second ago. Correspondingly, the preset head angle threshold may be the maximum angle used to characterize a change in the user's head orientation. That is, if the front-to-back head orientation angle of the user exceeds the preset head angle threshold, it indicates that the user's head orientation has changed at this time, otherwise it indicates that the user's head orientation has not changed.

[0141] It should be noted that the above attention state may be a feature used to characterize the degree of concentration of the user's attention. In this embodiment, the larger the user's attention state value is, the more concentrated the user's attention is, otherwise, the more scattered the user's attention is.

[0142] In practical applications, refer to Fig.10 , Fig.10 This is a flow chart for determining the user's attention state in the mixed reality interaction method based on human factor computing in this application. Fig.10 As shown, in this embodiment, the VR head display can be used to capture the head orientation, and the degree of concentration can be determined by the frequency of head orientation changes over a period of time. For example, in this embodiment, the time capture interval can be set to 1s, and the time judgment window can be set to 5s to determine the user's attention state. Specifically, the angle α between the head orientation of each second and the previous second can be averaged within a time judgment window (5s) to obtain the attention state (0-1), that is, Fig.10 Output in output = avg(β i , β i+1 , β i+2 , β i+3 , β i+4 ).

[0143] It should be noted that the above-mentioned contralateral height of the space is controlled by the attention state, and the attention state is quantified as a value in the range of 0 to 1 in the behavior calculation. The higher the attention state value, the lower the contralateral space height. In this embodiment, the spatial area that needs to change can be first determined by the user's position. There are 18 vertices controlling the top surface of the space. In this embodiment, the distance between the coordinate point of the human head and the top surface vertices can be calculated and sorted, and the 8 points with the largest distance can be selected as the contralateral points of the space. Fig.11 , Fig.11 This is a schematic diagram of the change of the height of the opposite side of the space in the mixed reality interaction method based on human factor computing in this application. Fig.11As shown in the figure, when the attention state is within 0-0.25, the height of the opposite side of the space does not change; when the attention state is within 0.25-0.5, the height of the opposite side of the space drops by 0.3m; when the attention state is within 0.5-0.75, the height of the opposite side of the space drops by 0.7m; when the attention state is within 0.75-1, the height of the opposite side of the space drops by 1m. The transition between the four states is smooth. For example, when the attention state changes from 0.2 to 0.6, the opposite side of the space gradually drops by 0.7m from the highest point, and the change time is about 1 second.

[0144] It should be noted that the core point of this scheme is to drive the change of space through the behavior of the actors, which is divided into four parts: communication mode, task processing mode, attention state and emotional state. Fig.12 , Fig.12 This is a flow chart of the user behavior driving space changes in the mixed reality interaction method based on human factor computing in this application. Fig.12 As shown, in this embodiment, user behavior can be mapped to a number from 0 to 1, and then 0 to 1 is used to drive spatial changes. For the communication mode, the head orientation can be used to determine whether a person is in a communication state (0 / 1). Combined with the number of people, position and relative position, it affects the spatial enclosure form; for the task processing mode, after studying the various sitting postures of the user in this scene, the key joint angles can be extracted, and two sets of data are obtained by calculation, which respectively reflect the overall sitting flexibility of the person (0~1) and the thinking intensity of the person (0~1). The data will cause changes in spatial regularity and spatial openness respectively; for the attention state, the head orientation is obtained through VR, and the static state of the head orientation over a period of time is calculated as the attention state (0~1), thereby driving the height change on the opposite side of the space and creating a more open space; for the emotional state, the confidence levels of four emotions are captured through emotions, corresponding to four 0~1 data, driving the saturation of four colors (usually there is a dominant emotion in a scene).

[0145] In the specific implementation, refer to Fig.13 , Fig.13 The overall flow chart of the mixed reality interaction method based on human factor computing in this application is as follows: Fig.13As shown, first, the device can obtain the user's human skeleton information, including the coordinates of the skeleton points, and capture the user's emotional information in real time through the depth camera, and at the same time obtain the human head position and orientation information in real time through the VR headset (Meta Quest 3). Then, the device can determine whether there is a dominant dimension based on the emotional values ​​corresponding to each emotional dimension in the emotional information. If not, the spatial grid vertices are assigned white; if so, the emotional dimension with the largest emotional value is determined as the dominant dimension, that is, the target emotional dimension, and the grid vertices are assigned colors that match the target emotional dimension. At the same time, the saturation of the space is determined based on the target emotional value corresponding to the target emotional dimension, where the higher the target emotional value, the higher the color saturation of the spatial grid. At the same time, the device can determine the user's thinking intensity based on the human body joint angles (including the angle between the user's upper arm and forearm, and the angle between the upper arm and upper body), and determine the user's sitting flexibility based on the human body trunk angle (including the angle between the user's spine and thigh, and the angle between the thigh and calf). Then, the spatial regularity and spatial openness of the virtual reality space can be determined based on the user's thinking intensity and sitting flexibility, respectively. Among them, the user's thinking intensity affects the position of the vertex on the curve in the space, and then affects the spatial regularity. Specifically, the higher the user's thinking intensity, the flatter the spatial enclosure surface, and the higher the spatial regularity. Correspondingly, the lower the user's thinking intensity, the more uneven the spatial enclosure surface, and the lower the spatial regularity. The higher the user's sitting flexibility, the more the grid surfaces that constitute the space are reduced, and the higher the spatial openness. Correspondingly, the lower the user's sitting flexibility, the more closed the space, the more complete the grid surfaces, and the lower the spatial openness. In addition, the device can also determine the separation state of the user and the target user within a certain range according to the user's head coordinates, and determine the communication state between the user and the target user according to the head orientation and the angle between the coordinates, and then determine whether the virtual reality space is separated based on the number of people in the range, the separation state and the communication state between the user and the target user, and then determine the spatial enclosure form. Furthermore, the device can also determine the user's attention state based on the change in the user's head orientation. The attention state is quantified as a value in the range of 0 to 1 in the behavior calculation. The higher the attention state value, the lower the height of the opposite space. Finally, the device can perform parametric modeling in Rhino Grasshopper based on the spatial regularity, spatial openness, spatial enclosure form and the height of the opposite side of the space based on the spatial color information of the virtual reality space, obtain a Rhino mesh model, and pass the final Rhino mesh model to Unity to present a real-scale space in the VR headset, thereby realizing human-computer interaction.

[0146] In this embodiment, it is disclosed that a first judgment result is obtained by judging whether a target user exists within a preset range of a user; if the first judgment result is existence, whether the user and the target user are on the same side is judged based on a preset separation position to obtain a second judgment result; a first head orientation corresponding to the user is determined based on the head position information of the user; a communication mode between the user and the target user is determined based on the first head orientation and a second head orientation corresponding to the target user; a spatial enclosure form of a virtual reality space is determined based on the first judgment result, the second judgment result and the communication mode; and a height of the opposite side of the space of the virtual reality space is determined based on the head position information, thereby satisfying the user's communication and solitude needs based on the user's communication mode, thereby improving the intelligence of human-computer interaction.

[0147] This application also provides a mixed reality interaction device based on human factor computing, please refer to Fig.14 , the mixed reality interaction device based on human factors computing includes:

[0148] A color information determination module 10, used to determine the spatial color information of the virtual reality space according to the user's emotional information;

[0149] A spatial information determination module 20, configured to determine the spatial regularity and spatial openness of the virtual reality space based on the human skeleton information of the user;

[0150] The spatial information determination module 20 is further used to determine the spatial enclosure form of the virtual reality space based on the head position information of the user;

[0151] A model building module 30, for building a Rhino grid model based on the spatial color information, the spatial regularity, the spatial openness and the spatial enclosure form;

[0152] The human-computer interaction module 40 is used to display the virtual reality space in a head display device through the Rhino grid model to interact with the user in real time.

[0153] The mixed reality interaction device based on human factors computing provided by the present application adopts the mixed reality interaction method based on human factors computing in the above-mentioned embodiment, which can solve the technical problem that the mixed reality human-computer interaction method in the prior art is difficult to capture and tap into the subconscious needs of users, resulting in the inability to provide intelligent services to users. Compared with the prior art, the beneficial effects of the mixed reality interaction device based on human factors computing provided by the present application are the same as the beneficial effects of the mixed reality interaction method based on human factors computing provided by the above-mentioned embodiment, and the other technical features in the mixed reality interaction device based on human factors computing are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0154] The present application provides a mixed reality interaction device based on human factors computing, and the mixed reality interaction device based on human factors computing includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the mixed reality interaction method based on human factors computing in the above-mentioned embodiment one.

[0155] Reference below Fig.15 , which shows a schematic diagram of the structure of a mixed reality interaction device based on human factors computing suitable for implementing the embodiment of the present application. The mixed reality interaction device based on human factors computing in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.15 The mixed reality interaction device based on human factors computing shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0156] like Fig.15As shown, the mixed reality interaction device based on human factors computing may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the mixed reality interaction device based on human factors computing are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the mixed reality interaction device based on human factors computing to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a mixed reality interaction device based on human factors computing with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0157] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the mixed reality interaction method based on human factors computing in the above-mentioned embodiment.

[0158] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned mixed reality interaction method based on human factors computing, and can solve the technical problem that the mixed reality human-computer interaction method in the prior art is difficult to capture and tap the subconscious needs of users, resulting in the inability to provide intelligent services to users. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the mixed reality interaction method based on human factors computing provided by the above-mentioned embodiments, and will not be repeated here.

[0159] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A mixed reality interaction method based on human factors computing, characterized in that: The method includes: Determine spatial color information of the virtual reality space according to the user's emotional information; Determining the spatial regularity and spatial openness of the virtual reality space based on the human skeleton information of the user; Determining a spatial enclosure form of the virtual reality space based on the head position information of the user; Constructing a Rhino grid model based on the spatial color information, the spatial regularity, the spatial openness and the spatial enclosure form; The virtual reality space is displayed in a head display device through the Rhino grid model to interact with the user in real time.

2. The method according to claim 1, characterized in that The step of determining the spatial color information of the virtual reality space according to the user's emotional information comprises: Collecting the user's facial expressions, and obtaining the user's emotional information based on the facial expressions through an emotion recognition algorithm, wherein the emotional information includes several emotional dimensions; Quantifying each emotion dimension to obtain an emotion value corresponding to the emotion dimension; The spatial color information of the virtual reality space is determined based on the emotion value.

3. The method according to claim 2, characterized in that The step of determining the spatial color information of the virtual reality space based on the emotion value comprises: Determine a target emotion dimension from the emotion dimensions based on the emotion value, the target emotion dimension being the emotion dimension with the largest emotion value among the emotion dimensions; Determining the spatial color saturation of the virtual reality space according to the target emotion value corresponding to the target emotion dimension; Determining the spatial hue of the virtual reality space according to the matching color corresponding to the target emotion value; Spatial color information of the virtual reality space is determined based on the spatial color saturation and the spatial hue.

4. The method according to claim 1, characterized in that The step of determining the spatial regularity and spatial openness of the virtual reality space based on the human skeleton information of the user comprises: Determine a first angle, a second angle, a third angle and a fourth angle corresponding to the user based on the human skeleton information of the user, wherein the first angle is the angle between the spine and the thigh of the user, the second angle is the angle between the thigh and the calf of the user, the third angle is the angle between the upper arm and the forearm of the user, and the fourth angle is the angle between the upper arm and the upper body of the user; determining the user's sitting flexibility according to the first angle and the second angle; Determining the thinking intensity of the user according to the third angle and the fourth angle; Determining the spatial openness of the virtual reality space based on the sitting posture flexibility of the user; The spatial regularity of the virtual reality space is determined based on the thinking intensity of the user.

5. The method according to claim 1, characterized in that The step of determining the spatial enclosure form of the virtual reality space based on the head position information of the user comprises: Determine whether a target user exists within a preset range of the user, and obtain a first determination result; If the first judgment result is yes, judging whether the user and the target user are on the same side based on a preset separation position, and obtaining a second judgment result; Determining a first head orientation corresponding to the user based on the head position information of the user; determining a communication mode between the user and the target user based on the first head orientation and a second head orientation corresponding to the target user; The spatial enclosure form of the virtual reality space is determined based on the first judgment result, the second judgment result and the interaction mode.

6. The method according to claim 5, characterized in that The step of determining the interaction mode between the user and the target user based on the first head orientation and a second head orientation corresponding to the target user comprises: Acquire a first connection angle between the first head orientation and a head line, and a second connection angle between a second head orientation corresponding to the target user and the head line, the head line being a line between the heads of the user and the target user; The interaction mode between the user and the target user is determined based on the first connection angle and the second connection angle.

7. The method according to claim 5, characterized in that After the step of determining the spatial enclosure form of the virtual reality space based on the first judgment result, the second judgment result and the communication mode, the method further includes: When the virtual reality space is in a spatially enclosed form, determining a front and rear head orientation angle of the user within a target time interval based on the head position information; Determining a frequency of a change in the user's head orientation according to the front-to-back head orientation angle and a preset head angle threshold; Determining the attention state corresponding to the user according to the frequency of the head orientation change; determining a spatially opposite height of the virtual reality space based on the attention state; The step of constructing a Rhino grid model based on the spatial color information, the spatial regularity, the spatial openness and the spatial enclosure form includes: A Rhino grid model is constructed based on the color information of the space, the regularity of the space, the openness of the space and the height of the opposite side of the space.

8. A mixed reality interaction device based on human factors computing, characterized in that: The device comprises: A color information determination module, used to determine the spatial color information of the virtual reality space according to the user's emotional information; A spatial information determination module, used to determine the spatial regularity and spatial openness of the virtual reality space based on the human skeleton information of the user; The spatial information determination module is further used to determine the spatial enclosure form of the virtual reality space based on the head position information of the user; A model building module, used for building a Rhino grid model based on the spatial color information, the spatial regularity, the spatial openness and the spatial enclosure form; A human-computer interaction module is used to display the virtual reality space in a head display device through the Rhino grid model to interact with the user in real time.

9. A mixed reality interactive device based on human factors computing, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the mixed reality interaction method based on human factors computing as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the mixed reality interaction method based on human factors computing as described in any one of claims 1 to 7 are implemented.

Images