Extended reality-based interaction method and apparatus, device, and medium

Abstract

Embodiments of the present disclosure relate to an extended reality-based interaction method and apparatus, a device, and a medium. The method comprises: first, acquiring node data of a target human joint point, wherein the node data of the target human joint point comprises one or more human joint points in a preset human skeleton model, and the node data is determined on the basis of tracking data of a target tracking device; and then when it is determined on the basis of the node data of the target human joint point that a preset interaction condition is currently met, displaying a target virtual resource in an extended reality space. Since node data of human joint points can more accurately represent interaction behaviors of a user, according to the embodiments of the present disclosure, whether the preset interaction condition in the extended reality space is currently met is determined on the basis of the node data of the target human joint point, and the accuracy of identifying interaction behaviors of users can be improved, thereby improving the authenticity of user interaction in an extended reality space.

Description

Interaction method, device, equipment and medium based on extended reality

[0001] This application claims priority to the Chinese invention patent application entitled “Interaction method, device, equipment and medium based on extended reality” and application number 202410132309.X, filed on January 30, 2024. The entire contents of this application are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of virtual display technology, and in particular to an interactive method, apparatus, device, and medium based on extended reality. Background Art

[0003] Extended Reality (XR) technology combines the real and virtual through computers, creating an extended reality space that allows for human-computer interaction. XR is a general term for various technologies, including augmented reality (AR), virtual reality (VR), and mixed reality (MR). By integrating the visual interaction technologies of these three, users can interact with the virtual world in an immersive way.

[0004] In related technologies, a handle is usually used as the main extended reality device to complete the interaction with the virtual world. Specifically, based on the collected handle posture data, it is judged whether the user has performed a certain interactive behavior, and there may be a problem of insufficient accuracy in identifying the interactive behavior.

[0005] Therefore, how to improve the accuracy of interactive behavior recognition to enhance the authenticity of user interactions in the extended reality space is a technical problem that needs to be solved urgently. Summary of the Invention

[0006] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides an interactive method, device, equipment and medium based on extended reality, which can improve the accuracy of identifying user interactive behaviors, thereby improving the authenticity of user interactions in the extended reality space.

[0007] In a first aspect, an embodiment of the present disclosure provides an interaction method based on extended reality, the method comprising: obtaining node data of a target human joint point; wherein the target human joint point includes one or more human joint points in a preset human skeleton model, and the node data is determined based on tracking data of a target tracking device; when it is determined according to the node data of the target human joint point that a preset interaction condition is currently met, the target virtual resource is displayed in the extended reality space.

[0008] In an optional implementation, obtaining the node data of the target human joint point includes: obtaining the node data of the human joint point corresponding to the target interactive object in the extended real space.

[0009] In an optional embodiment, the target human joint points include multiple human joint points, the preset interaction conditions include joint point distribution conditions, and determining whether the preset interaction conditions are currently satisfied based on the node data of the target human joint points includes: determining that the distribution data of the multiple human joint points satisfies the joint point distribution conditions in the preset interaction conditions.

[0010] In an optional embodiment, the preset interaction conditions include interaction time range conditions, and determining whether the preset interaction conditions are currently satisfied based on the node data of the target human joint points includes: determining that the timestamp information of the node data of the target human joint points satisfies the interaction time range conditions in the preset interaction conditions.

[0011] In an optional embodiment, obtaining the node data of the target human joint points includes: obtaining sensor data and / or image data of the target tracking device; determining the tracking data of the target tracking device based on the sensor data and / or the image data; and determining the node data of the target human joint points based on the tracking data of the target tracking device.

[0012] In an optional embodiment, determining the node data of the target human joint points based on the tracking data of the target tracking device includes: using a target algorithm model to determine the node data of the target human joint points based on the tracking data of the target tracking device; wherein, the target algorithm model is trained using data of the preset human skeleton model.

[0013] In an optional embodiment, the target tracking device includes at least a head-mounted device, and the node data of the target human joint point is relative position node data determined based on a root node, and the root node is determined based on the tracking data of the head-mounted device.

[0014] In an optional embodiment, the node data includes position information, the preset interaction conditions include preset interaction position information, and determining whether the preset interaction conditions are currently satisfied based on the node data of the target human joint point includes: determining that the position information of the target human joint point matches the preset interaction position information in the preset interaction conditions.

[0015] In an optional embodiment, the node data also includes speed information, and the preset interaction conditions also include preset speed information corresponding to the preset interaction position information. Determining whether the preset interaction conditions are currently met based on the node data of the target human joint point includes: determining that the speed information of the target human joint point matches the preset speed information in the preset interaction conditions.

[0016] In an optional implementation, the preset interaction condition also includes a preset interaction action condition, and before displaying the target virtual resource in the extended real space, it also includes: identifying the target interaction action; and determining that the target interaction action satisfies the preset interaction action condition.

[0017] In an optional embodiment, the identifying target interaction action includes: identifying the target interaction action based on tracking data of the target tracking device and a first action recognition model; or identifying the target interaction action based on the node data and a second action recognition model.

[0018] In an optional embodiment, the target tracking device includes at least a lower limb tracking device.

[0019] In an optional implementation, the target tracking device further includes a head-mounted device, or a head-mounted device and a handle device.

[0020] In an optional embodiment, the target tracking device further includes a head-mounted device, and the node data is determined based on tracking data and hand tracking data of the target tracking device, and the hand tracking data is acquired using the head-mounted device.

[0021] In a second aspect, the present disclosure provides an interactive device based on extended reality, the device comprising: an acquisition module for acquiring node data of a target human joint point; wherein the target human joint point includes one or more human joint points in a preset human skeleton model, and the node data is determined based on tracking data of a target tracking device; a first determination module for determining, based on the node data of the target human joint point, that when a preset interaction condition is currently met, displaying the target virtual resource in the extended reality space.

[0022] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; and the processor for reading the executable instructions from the memory and executing the instructions to implement an extended reality-based interaction method as provided in an embodiment of the present disclosure.

[0023] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the interaction method based on extended reality as provided in an embodiment of the present disclosure.

[0024] In a fifth aspect, the present disclosure provides a computer program product, which includes a computer program / instructions. When the processor executes the computer program / instructions, the above method is implemented.

[0025] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art:

[0026] In the interactive method based on extended reality provided by the embodiment of the present disclosure, first, the node data of the target human joint point is obtained, wherein the node data of the target human joint point includes one or more human joint points in a preset human skeleton model, and the node data is determined based on the tracking data of the target tracking device; then, when it is determined based on the node data of the target human joint point that the preset interaction conditions are currently met, the target virtual resource is displayed in the extended reality space. Since the node data of the human joint point can more accurately characterize the user's interactive behavior, the embodiment of the present disclosure determines whether the preset interaction conditions in the extended reality space are currently met through the node data of the target human joint point, which can improve the accuracy of the recognition of the user's interactive behavior, thereby improving the authenticity of the user's interaction in the extended reality space. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0028] FIG1 is a schematic diagram of an application scenario of a virtual reality device provided by an embodiment of the present disclosure;

[0029] FIG2 is a flow chart of an interactive method based on extended reality provided by an embodiment of the present disclosure;

[0030] FIG3 is a schematic diagram of a preset human skeleton model provided by an embodiment of the present disclosure;

[0031] FIG4 is a schematic structural diagram of an interactive device based on extended reality according to an embodiment of the present disclosure; and

[0032] FIG5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0034] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0035] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0036] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0037] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0038] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0039] Some technical concepts or noun concepts involved in this article are explained:

[0040] Extended reality devices, terminals that achieve extended effects, can usually be provided in the form of glasses, helmet-mounted displays (HMDs), or contact lenses to achieve visual perception and other forms of perception. Of course, the form of extended reality devices is not limited to this and can be further miniaturized or enlarged as needed.

[0041] Virtual reality (VR) technology, also known as virtual environment, spiritual environment or artificial environment, refers to the technology of using computers to generate a virtual world that can directly exert visual, auditory and tactile sensations on participants and allow them to observe and operate interactively.

[0042] Augmented Reality (AR) technology refers to the technology of using computers to integrate virtual information with the real world. It widely uses a variety of technical means such as multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, and sensing. It simulates computer-generated virtual information such as text, images, three-dimensional models, music, and videos, and applies them to the real world, thereby achieving "enhancement" of the real world.

[0043] The extended reality devices described in the embodiments of the present invention may include but are not limited to the following types:

[0044] Computer-based virtual reality (PCVR) devices use the PC to perform calculations and data output related to virtual reality functions. External computer-based virtual reality devices use the data output by the PC to achieve virtual reality effects.

[0045] Mobile virtual reality devices support the configuration of mobile terminals (such as smartphones) in various ways (such as head-mounted displays with dedicated card slots). Through wired or wireless connection with the mobile terminal, the mobile terminal performs relevant calculations for virtual reality functions and outputs data to the mobile virtual reality device, such as watching virtual reality videos through the mobile terminal's APP.

[0046] All-in-one virtual reality devices have a processor for performing related calculations for virtual functions, and therefore have independent virtual reality input and output functions. They do not need to be connected to a PC or mobile terminal and have a high degree of freedom in use.

[0047] Virtual reality objects are objects that interact in virtual scenes (such as target interactive objects in extended reality space), which are controlled by users or robot programs (for example, artificial intelligence-based robot programs) and can be still, move, and perform various behaviors in virtual scenes, such as virtual people corresponding to users in live broadcast scenes and obstacle walls in extended reality games.

[0048] As shown in Figure 1, the HMD is relatively lightweight, ergonomically comfortable, and provides high-resolution content with low latency. Virtual reality devices are equipped with posture detection sensors (such as a nine-axis sensor) to monitor the device's posture changes in real time. When a user wears a virtual reality device, any changes in their head posture transmit this real-time posture information to a processor, which calculates the user's gaze point in the virtual environment. Based on this gaze point, an image within the user's gaze range (i.e., virtual field of view) within the 3D model of the virtual environment is calculated and displayed on the display, creating an immersive experience that feels like viewing the real world.

[0049] In actual applications, when a user wears an HMD device and opens a predetermined application, such as an XR dance game, the HMD device will run a corresponding virtual scene (also known as an extended reality space). The virtual scene can be a simulation of the real world, a semi-simulated and semi-fictional virtual scene, or a purely fictional virtual scene. The virtual scene can be any of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, or a three-dimensional virtual scene. The embodiment of the present application does not limit the dimensions of the virtual scene. For example, a virtual scene may include characters, sky, land, ocean, etc., and the land may include environmental elements such as deserts and cities. Users can control virtual objects to move in the virtual scene, and can also interactively control controls, models, display content, characters, etc. in the virtual scene through handle devices, bare-hand gestures, etc.

[0050] Based on the above, in the extended reality space, when judging whether the user has performed a certain interactive behavior based only on the collected handle posture data, there may be a problem of insufficient accuracy in identifying the user's interactive behavior, affecting the authenticity of the user's interaction in the extended reality space.

[0051] To enhance the authenticity of user interactions in an extended reality space, the disclosed embodiments provide an extended reality-based interaction method. First, node data for a target human joint is acquired, where the node data for the target human joint includes one or more human joints in a preset human skeletal model and is determined based on tracking data from a target tracking device. Then, when a preset interaction condition is determined to be met based on the node data for the target human joint, a target virtual resource is displayed in the extended reality space.

[0052] Since the node data of human joints can more accurately characterize the user's interactive behavior, the embodiment of the present disclosure uses the node data of the target human joints to determine whether the preset interaction conditions in the extended reality space are currently met, which can improve the accuracy of identifying the user's interactive behavior, thereby improving the authenticity of the user's interaction in the extended reality space.

[0053] Based on this, an embodiment of the present disclosure provides an interactive method based on extended reality, which is introduced below in conjunction with specific embodiments.

[0054] FIG2 is a flow chart of an interactive method based on extended reality provided by an embodiment of the present disclosure, which can be executed by an extended reality device. As shown in FIG2 , the method includes: S201 : obtaining node data of a target human joint point.

[0055] The target human joint points include one or more human joint points in a preset human skeleton model, and the node data is determined based on tracking data of a target tracking device.

[0056] The extended reality interaction method provided in the embodiments of the present disclosure can be applied to an extended reality device. The extended reality device can be a head-mounted device, such as a helmet-mounted display (HMD), or other types of extended reality devices, which are not limited in the embodiments of the present disclosure.

[0057] In the disclosed embodiments, node data of a target human joint is determined based on tracking data from a target tracking device. Specifically, the tracking data may include position data and / or posture data of the target tracking device. In practical applications, the target tracking device can be tracked and positioned based on the position data and / or posture data of the target tracking device. The specific method for determining the node data of a target human joint based on the tracking data of the target tracking device will be described in detail in subsequent embodiments and is not further elaborated here.

[0058] In an optional embodiment, the target tracking device may include at least a lower limb tracking device, such as a leg tracking device, a foot tracking device, etc. In actual applications, the lower limb tracking device may be installed with multiple light-emitting elements, such as infrared LED lamps. By collecting images of the light-emitting elements when they are emitting light, the lower limb tracking device is optically positioned by optical means, and then the tracking data of the lower limb tracking device is calculated.

[0059] In another optional embodiment, the target tracking device may further include a lower limb tracking device and a head-mounted device. The head-mounted device may be, for example, glasses, a helmet-mounted display (HMD), contact lenses, or the like, for enabling visual perception and other forms of perception. Node data may be determined based on tracking data from the target tracking device and hand tracking data, wherein the hand tracking data is data acquired using the head-mounted device.

[0060] In another optional embodiment, the target tracking device may also include at least a lower limb tracking device, a head-mounted device and a handle device. Based on the tracking data of the lower limb tracking device, the head-mounted device and the handle device, the node data of the target human joint point can be determined.

[0061] In the embodiment of the present disclosure, the preset human skeleton model may define 24 human joints and the positional relationships between the joints. FIG3 is a schematic diagram of a preset human skeleton model provided in the embodiment of the present disclosure.

[0062] In actual applications, since there is a positional relationship between the 24 human joints in the preset human skeleton model, the node data of the 24 human joints can be calculated based on the tracking data of the target tracking device, such as the node data of the foot joint 11 and the hand joint 23.

[0063] Specifically, the target tracking device can be bound to a preset part of the human body, such as the head, legs, hands, etc. After obtaining the tracking data of the target tracking device, the tracking data of the target tracking device is input into a preset human skeleton model. The preset human skeleton model will solve and process the tracking data of the target tracking device based on the positional relationship between the human joints, and output the node data of 24 human joint points.

[0064] It is worth noting that the preset human skeleton model in the embodiment of the present disclosure not only includes a human skeleton model with 24 human joints defined, but also includes a human skeleton model with other numbers of human joints defined, such as a preset human skeleton model with 20 human joints defined, etc. The embodiment of the present disclosure does not impose specific restrictions on the number of human joints in the preset human skeleton model and the positional relationship between the various human joints.

[0065] In the embodiment of the present disclosure, the target human joint point may include one or more human joint points in a preset human skeleton model. For example, the target human joint point may be 24 human joint points in the preset human skeleton model, or lower limb joint points, etc.

[0066] In an optional embodiment, the target human joint points may include the human joint points corresponding to the target interactive objects in the extended reality space, and the node data of the target human joint points may include the node data of the human joint points corresponding to the target interactive objects in the extended reality space, wherein the target interactive objects may be any one or more interactive objects in the extended reality space, that is, virtual interactive objects in the virtual scene. For example, in XR music games, the target interactive objects may include virtual interactive objects such as "virtual notes" and "virtual plots".

[0067] S202: When it is determined based on the node data of the target human joint point that a preset interaction condition is currently satisfied, the target virtual resource is displayed in the extended real space.

[0068] In actual applications, after determining the node data of the target human joint points based on the tracking data of the target tracking device, it is determined whether the preset interaction conditions are currently met based on the node data of the target human joint points. When the preset interaction conditions are met, the target virtual resources are displayed in the extended real space to indicate that the user's interaction in the extended real space is successful.

[0069] For example, the target virtual resource can be used to present the effect of successful interaction with the target interactive object, for example, the target virtual resource can be the sound effect of the knee hitting the note, the animation effect of the foot stepping on the note, etc.

[0070] In actual applications, different target interaction objects can correspond to different preset interaction conditions. For example, assuming that the target interaction object is a virtual note in the extended real space, the corresponding preset interaction condition is that the position of the user's foot joint is within the preset position range. It can be understood that the human body joint corresponding to the target interaction object at this time is the foot joint. Accordingly, the node data of the target human body joint obtained is the node data corresponding to the foot joint, such as the position information of the foot joint.

[0071] In the disclosed embodiment, the preset interaction conditions are used to match the node data of the target human joint point to determine whether the user interaction behavior reflected by the node data satisfies the preset interaction conditions. The preset interaction conditions may include interaction conditions of various dimensions, such as a combination of one or more of the following: joint point distribution conditions, interaction time range conditions, interaction position information, speed information, and interaction action conditions.

[0072] In the interactive method based on extended reality provided by the embodiment of the present disclosure, first, the node data of the target human joint point is obtained, wherein the node data of the target human joint point includes one or more human joint points in a preset human skeleton model, and the node data is determined based on the tracking data of the target tracking device; then, when it is determined based on the node data of the target human joint point that the preset interaction conditions are currently met, the target virtual resource is displayed in the extended reality space. Since the node data of the human joint point can more accurately characterize the user's interactive behavior, the embodiment of the present disclosure determines whether the preset interaction conditions in the extended reality space are currently met through the node data of the target human joint point, which can improve the accuracy of the recognition of the user's interactive behavior, thereby improving the authenticity of the user's interaction in the extended reality space.

[0073] In practical applications, the node data of the target human joint point can be determined based on tracking data of a target tracking device, wherein the tracking data of the target tracking data can be obtained based on sensor data and image data of the target tracking device.

[0074] In the embodiment of the present disclosure, first, sensor data and / or image data of the target tracking device are obtained, then, tracking data of the target tracking device is determined based on the sensor data and / or image data, and then, based on the tracking data of the target tracking device, node data of the target human joint point is determined.

[0075] In actual applications, target tracking devices may include lower limb tracking devices and head-mounted devices. Among them, the lower limb tracking devices may be equipped with IMU sensors. IMU sensors, also known as inertial sensors, are a combination of accelerometers and gyroscope sensors. They are used to detect the acceleration and angular velocity of the target tracking device to represent its movement, such as six-axis IMU sensors, nine-axis IMU sensors, etc.

[0076] In an embodiment of the present disclosure, the lower limb tracking device can collect sensor data of the lower limb tracking device based on its own built-in IMU sensor and report it to the head-mounted device, so that the head-mounted device can determine the tracking data of the lower limb tracking device based on the received sensor data; the lower limb tracking device can also locally calculate the tracking data based on the sensor data and report it to the head-mounted device.

[0077] In actual applications, the lower limb tracking device can also be provided with a shooting component. The lower limb tracking device can collect image data of the surrounding environment through the shooting component and upload the image data to the head-mounted device, so that the head-mounted device can determine the tracking data of the lower limb tracking device based on the received image data; the lower limb tracking device can also locally calculate the tracking data based on the image data of the surrounding environment and upload it to the head-mounted device.

[0078] In an optional embodiment, the image data of the lower limb tracking device may also include image data collected by other associated devices (such as a head-mounted device or other external cameras).

[0079] In the embodiment of the present disclosure, after obtaining sensor data and / or image data of the target tracking device, tracking data of the target tracking device can be determined based on the sensor data and / or image data. The tracking data of the target tracking device may include position data and posture data of the target tracking device.

[0080] In practical applications, the extended reality device can collect image data from the target tracking device's light-emitting element, optically locate the target tracking device, and then calculate the position data and posture data of the target tracking device. The disclosed embodiments can also calculate the position data and posture data of the target tracking device based on other methods, which are not detailed here.

[0081] In an optional implementation, in order to further improve the recognition accuracy of user interaction behavior reflected by node data, after determining the tracking data of the target tracking device, the target algorithm model can be used to determine the node data of the target human joint point based on the tracking data of the target tracking device.

[0082] In the embodiment of the present disclosure, the target algorithm model can be trained using data that matches a preset human skeleton model. The preset human skeleton model can include 24 human joints. The target algorithm model can process the tracking data of the target tracking device based on the preset human skeleton model to obtain node data of the target human joints.

[0083] In an embodiment of the present disclosure, the input of the target algorithm model may include tracking data of the target tracking device, for example, tracking data of a lower limb tracking device, posture data of a head-mounted device and tracking data of a lower limb tracking device, posture data of a head-mounted device, posture data of a hand tracking device and tracking data of a lower limb tracking device, etc.

[0084] In an optional embodiment, the input data of the target algorithm model may also include tracking data of the target tracking device and other tracking data. The other tracking data may include, for example, hand tracking data, etc., wherein the hand tracking data may be obtained using a head-mounted device.

[0085] Exemplarily, first, the tracking data of the target tracking device is input into the target algorithm model, and then the node data of the human joint points that match the preset human skeleton model is output. Then, the node data of the target human joint points are obtained from the node data of the human joint points of the preset human skeleton model.

[0086] In the embodiment of the present disclosure, the node data of the human joints of the preset human skeleton model output may include position data and speed data of each human joint, wherein the position data refers to the x, y, and z coordinate data of the human joints, and the speed data may be calculated based on the position information of the human joints in one frame or multiple frames of sensor data and / or image data.

[0087] In another optional embodiment, the target tracking device may also include at least a head-mounted device. Accordingly, the node data of the target human joint point may also be relative position node data determined based on a root node, wherein the root node is determined based on the tracking data of the head-mounted device.

[0088] For example, taking the head joint as the root node (or upper node) and other human joints as subordinate nodes (or lower nodes) of the root node, the node data of the human joints in the preset human skeleton model may include the posture data (i.e., position data and posture data) of the root node (or upper node) and the posture data of each subordinate node relative to the root node (or upper node).

[0089] Specifically, the tracking data of the head-mounted device is first determined as the node data of the head joint point in the preset human skeleton model, and the head joint point is used as the root node; then, based on the positional relationship between the human joint points in the preset human skeleton model and the node data of the root node, the node data of the human joint points in the preset human skeleton model is determined; finally, the node data of the target human joint point is obtained from the node data of the human joint points in the preset human skeleton model.

[0090] In order to enrich the user's interaction methods in the extended reality space, after obtaining multiple human joint points, it is also possible to determine whether the distribution data of multiple human joint points meet the joint point distribution conditions in the preset interaction conditions. If it is determined that the distribution data of multiple human joint points meet the joint point distribution conditions in the preset interaction conditions, the target virtual resources are displayed in the extended reality space.

[0091] In the embodiment of the present disclosure, distribution data refers to the relative position distribution relationship between multiple human joints, and the joint distribution conditions are used to characterize the preset interaction position information of multiple human joints. Accordingly, determining whether the distribution data of multiple human joints meet the joint distribution conditions can include determining whether multiple human joints respectively match their respective preset interaction position information.

[0092] In an optional embodiment, the distribution data can also be used to represent the position distribution of multiple human joints in the extended reality space, and the joint distribution conditions can be used to characterize the position distribution relationship of multiple human joints in the extended reality space. For example, the foot joints and hand joints are respectively distributed at the preset upper position and the preset lower position in the extended reality space.

[0093] To facilitate understanding of the embodiments of the present application, taking sports fitness games or leg dances as an example, first, the node data of the foot joints and the node data of the knee joints are obtained, and then, it is determined whether the position distribution of the foot joints and the knee joints meets the joint distribution conditions, for example, whether the position distribution of the foot joints and the knee joints is at the preset positions. If it is determined that the position distribution of the foot joints and the knee joints is within the preset positions, it means that the distribution data of the foot joints and the knee joints meet the joint distribution conditions. At this time, the target virtual resources are displayed in the extended reality space, such as the dynamic effect of the knee hitting the musical notes, so as to enhance the user's immersion in the expanded space.

[0094] In order to further enrich the user's interaction methods in the extended reality space, the target virtual resource can also be displayed in the extended reality space when it is determined that the timestamp information of the node data of the target human joint point meets the interaction time range condition in the preset interaction condition.

[0095] In the embodiment of the present disclosure, since the node data of the target human joints is determined based on the tracking data of the target tracking device, and the tracking data is determined based on the sensor data and / or image data, and the sensor data and / or image data carry timestamp information, the node data of the target human joints may include timestamp information, and the timestamp information of the node data may be used to characterize the time information of the target human joints. The interaction time range condition may define the time range information of the target interactive object that can interact in the extended real space. For example, within a preset time range after the note or plot lights up (i.e., within the time range within which the note or plot can interact), it is determined whether the interaction time range condition is currently met based on the timestamp information of the target human joints.

[0096] To facilitate understanding of the embodiments of the present application, taking a dance machine game as an example, within a preset time range after the ground block lights up, determine whether the node data of the foot joint points matches other preset interaction conditions, such as determining whether the position information of the foot joint points matches the preset interaction position information; if it is determined that within the preset time range, the node data of the foot joint points matches the preset interaction position information in the preset interaction conditions, it means that the user has stepped on the bright block, and the target virtual resources are displayed in the extended reality space, such as the animation effects of stepping on the bright block.

[0097] In an optional embodiment, if the node data of the foot joint point does not match any preset interaction condition within a preset time range, that is, it is determined that the timestamp information of the foot joint point does not meet the interaction time range condition, it means that the user has not stepped on the bright block. At this time, preset virtual resources are displayed in the extended reality space, such as the animation effect of "missing the bright block", etc., wherein the preset virtual resources can be used to represent the user's failure to interact in the extended reality space.

[0098] In order to further enrich the user's interaction methods in the extended real space, it is also possible to determine whether to display the target resource object in the extended real space based on the position information of the target human joint point and the preset interaction position information in the preset interaction conditions.

[0099] In the embodiment of the present disclosure, the position information of the target joint point may include the coordinate information of the target human body joint point in the extended real space, and the preset interaction position information may define the position information of the target interaction object in the extended real space, such as the orbital position of the target interaction object.

[0100] To facilitate understanding of the embodiments of the present application, taking a music game involving leg movements as an example, determine whether the position information of the foot joint point (i.e., the target human joint point) matches the position information corresponding to the ground block (i.e., the target interactive object). If it is determined that the position information of the foot joint point matches the position information of the ground block, it means that the position information of the target human joint point matches the preset interactive position information in the preset interactive condition. At this time, the target virtual resources are displayed in the extended reality space, such as the music effects of stepping on the bright block, to prompt the user to step on the bright block.

[0101] In practical applications, sports and fitness games require users to actively interact with target objects using body parts. For example, to complete the game, users need to use their knee joints to hit musical notes. During the game, the user may raise their knees and wait for the virtual notes to hit them. Using only the position information of the knee joint to identify whether the user's interaction behavior meets the preset interaction conditions is obviously inaccurate. Therefore, in addition to identifying user interaction behavior based on position information, in order to further improve the recognition accuracy of user interaction behavior reflected by node data, the user's interaction behavior can also be identified in combination with the speed information included in the node data.

[0102] Specifically, it is determined whether the speed information of the target human joint point matches the preset speed information in the preset interaction condition. If it is determined that the speed information of the target human joint point matches the preset speed information in the preset interaction condition, the target virtual resource is displayed in the extended real space.

[0103] To facilitate understanding of the embodiments of the present application, taking sports and fitness games as an example, while determining that the position information of the knee joint point (i.e., the target human joint point) matches the preset interaction position information, it is also possible to determine whether the speed information of the knee joint point matches the preset speed information. If the speed information of the knee joint point matches the preset speed information, it means that the user has raised his knee and hit the target interactive object, and the target virtual resource is displayed in the extended reality space.

[0104] In order to further enrich the user's interaction methods in the extended reality space, in an optional implementation, it can also be determined whether to display the target resource object in the extended reality space based on the target interaction action and the preset interaction action conditions in the preset interaction conditions.

[0105] Among them, the target interaction action can be used to describe the relative position relationship of multiple human joints, such as knee strikes, marching kicks, squats or dance movements; the preset interaction conditions define the relative position relationship between multiple human joints, for example, the foot joint is within a preset range below the knee joint, or the angle of the foot joint relative to the knee joint is within a preset range.

[0106] In the embodiment of the present disclosure, a target interaction action is first identified, and then it is determined whether the target interaction action meets a preset interaction action condition. If so, the target virtual resource is displayed in the extended real space.

[0107] In practical applications, target interaction actions can be identified based on the tracking data of the target tracking device and the first action recognition model to improve the recognition accuracy of user interaction behaviors reflected by the node data. The first action recognition model can be trained based on the tracking data collected by the target tracking device when the target interaction action occurs, and the first action recognition model can be used to identify the target interaction action based on the tracking data of the target tracking device.

[0108] In actual applications, the training process of the first action recognition model may include: inputting training samples into the action recognition model, wherein the training samples may include sensor data corresponding to the target interaction action (such as stomping action) collected within a preset time period, and training the action recognition model based on the sensor data corresponding to the target interaction action to obtain a first action recognition model for identifying stomping actions.

[0109] During the application of the first action recognition model, first, the tracking data collected by the target tracking device is input into the first action recognition model corresponding to the target interaction action, and the recognition result is obtained after recognition processing by the first action recognition model; then, it is determined whether the recognition result includes the target interaction action. If it is determined that the recognition result includes the target interaction action, it means that the first action recognition model recognizes the target interaction action.

[0110] In another optional embodiment, the target interactive action can also be identified based on the node data and the second action recognition model. The second action recognition model can be used to identify the target interactive action based on the node data of the target human joint points.

[0111] Specifically, first, the node data of the acquired target human body node is input into the second action recognition model corresponding to the target interaction action, and the recognition result is obtained after recognition processing by the second action recognition model; then, it is determined whether the recognition result contains the target interaction action. If it is determined that the recognition result contains the target interaction action, it means that the second action recognition model recognizes the target interaction action.

[0112] In an optional implementation, after identifying the target interaction action, it is further determined whether the target interaction action meets the preset interaction action conditions, so that when the preset interaction action conditions are met, the target virtual resources corresponding to the preset interaction conditions are displayed in the extended reality space, thereby further enriching the user's interactive functions in the extended reality space.

[0113] In an optional implementation, the three dimensions of location information, speed information, and target interaction action in the node data may be combined to determine whether the preset interaction conditions are currently met, so as to further improve the recognition accuracy of the user interaction behavior reflected by the node data.

[0114] For example, in a fitness game, the target interactive object can be a preset area in the extended reality space, and the user uses their knee to interact with the target interactive object. During the game, the position of the user's knee joint and the preset area can be used to determine whether the user has hit the target interactive object.

[0115] While determining that the position of the knee joint meets the preset interaction position information, the positions of multiple joints can also be determined, such as whether the foot joint is at the preset position below the knee joint, and whether the actions reflected by the foot joint and the knee joint meet the preset interaction action conditions; or, while determining that the position of the knee joint meets the preset interaction position information, it can also be determined whether the speed information of the knee joint matches the preset speed information to determine whether the user hits the target interaction object.

[0116] In order to implement the above-mentioned embodiment, the present disclosure also proposes an interactive device based on extended reality. Figure 4 is a structural diagram of an interactive device based on extended reality provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated into an electronic device. As shown in Figure 4, the device includes: an acquisition module 401, which is used to obtain node data of a target human joint point; wherein the target human joint point includes one or more human joint points in a preset human skeleton model, and the node data is determined based on the tracking data of the target tracking device; a first determination module 402, which is used to determine that the preset interaction condition is currently met based on the node data of the target human joint point, and display the target virtual resource in the extended reality space.

[0117] In an optional implementation, the acquisition module includes: a first acquisition submodule, configured to acquire node data of human joint points corresponding to a target interactive object in an extended real space.

[0118] In an optional embodiment, the target human joint points include multiple human joint points, the preset interaction conditions include joint point distribution conditions, and the first determination module includes: a first determination submodule, used to determine whether the distribution data of the multiple human joint points meets the joint point distribution conditions in the preset interaction conditions.

[0119] In an optional embodiment, the preset interaction conditions include an interaction time range condition, and the first determination module further includes: a second determination submodule, used to determine whether the timestamp information of the node data of the target human joint point meets the interaction time range condition in the preset interaction conditions.

[0120] In an optional embodiment, the acquisition module includes: a second acquisition submodule, used to acquire sensor data and / or image data of the target tracking device; a third determination submodule, used to determine the tracking data of the target tracking device based on the sensor data and / or the image data; and a fourth determination submodule, used to determine the node data of the target human joint point based on the tracking data of the target tracking device.

[0121] In an optional embodiment, the fourth determination submodule is specifically used to: use the target algorithm model to determine the node data of the target human joint point based on the tracking data of the target tracking device; wherein, the target algorithm model is trained using the data of the preset human skeleton model.

[0122] In an optional embodiment, the target tracking device includes at least a head-mounted device, and the node data of the target human joint point is relative position node data determined based on a root node, and the root node is determined based on the tracking data of the head-mounted device.

[0123] In an optional embodiment, the node data includes position information, the preset interaction conditions include preset interaction position information, and the first determination module includes: a fifth determination submodule, used to determine whether the position information of the target human joint point matches the preset interaction position information in the preset interaction conditions.

[0124] In an optional embodiment, the node data also includes speed information, the preset interaction conditions also include preset speed information corresponding to the preset interaction position information, and the first determination module also includes: a fifth determination submodule, used to determine whether the speed information of the target human joint point matches the preset speed information in the preset interaction conditions.

[0125] In an optional embodiment, the preset interaction condition also includes a preset interaction action condition, and the device further includes: an identification module for identifying a target interaction action; and a second determination module for determining whether the target interaction action satisfies the preset interaction action condition.

[0126] In an optional embodiment, the recognition module includes: a first recognition submodule, used to identify the target interaction action based on the tracking data of the target tracking device and a first action recognition model; or, a first recognition submodule, used to identify the target interaction action based on the node data and a second action recognition model.

[0127] In an optional embodiment, the target tracking device includes at least a lower limb tracking device.

[0128] In an optional implementation, the target tracking device further includes a head-mounted device, or a head-mounted device and a handle device.

[0129] In an optional embodiment, the target tracking device further includes a head-mounted device, and the node data is determined based on tracking data and hand tracking data of the target tracking device, and the hand tracking data is acquired using the head-mounted device.

[0130] In the interactive device based on extended reality provided by the embodiment of the present disclosure, first, node data of the target human joint point is obtained, wherein the node data of the target human joint point includes one or more human joint points in a preset human skeleton model, and the node data is determined based on the tracking data of the target tracking device; then, when it is determined based on the node data of the target human joint point that the preset interaction condition is currently met, the target virtual resource is displayed in the extended reality space. Because the node data of the human joint point can more accurately characterize the user's interactive behavior, the embodiment of the present disclosure determines whether the preset interaction condition in the extended reality space is currently met through the node data of the target human joint point, which can improve the accuracy of the recognition of the user's interactive behavior, thereby improving the authenticity of the user's interaction in the extended reality space.

[0131] The extended reality-based interaction device provided in the embodiments of the present disclosure can execute the extended reality-based interaction method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0132] In order to implement the above embodiments, the present disclosure further proposes a computer program product, including a computer program / instruction, which implements the augmented reality-based interaction method in the above embodiments when executed by a processor.

[0133] FIG5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

[0134] 5 , which illustrates a schematic diagram of the structure of an electronic device 1000 suitable for implementing the embodiments of the present disclosure. The electronic device 1000 in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device illustrated in FIG5 is merely an example and should not limit the functionality or scope of use of the embodiments of the present disclosure.

[0135] As shown in FIG5 , the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the electronic device 1000 are also stored in the RAM 1003. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0136] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange data. Although FIG5 shows the electronic device 1000 with various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0137] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1009, or installed from the storage device 1008, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the interactive method based on extended reality of the embodiment of the present disclosure are performed.

[0138] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0139] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0140] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0141] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: first, obtains node data of a target human joint point, wherein the node data of the target human joint point includes one or more human joint points in a preset human skeleton model, and the node data is determined based on tracking data of a target tracking device; then, when it is determined based on the node data of the target human joint point that a preset interaction condition is currently met, the target virtual resource is displayed in the extended real space. Because the node data of the human joint point can more accurately represent the user's interactive behavior, the embodiment of the present disclosure determines whether the preset interaction condition in the extended real space is currently met based on the node data of the target human joint point, which can improve the accuracy of identifying the user's interactive behavior, thereby improving the authenticity of the user's interaction in the extended real space. The computer program code for performing the operations of the present disclosure can be written in one or more programming languages ​​or a combination thereof, and the programming languages ​​include but are not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, and also conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0143] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0144] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0145] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0146] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0147] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0148] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. An interactive method based on extended reality, comprising: Acquire node data of a target human joint point; wherein the target human joint point includes one or more human joint points in a preset human skeleton model, and the node data is determined based on tracking data of a target tracking device; When it is determined based on the node data of the target human joint point that a preset interaction condition is currently satisfied, the target virtual resource is displayed in the extended real space.

2. The method according to claim 1, wherein the step of obtaining the node data of the target human joint point comprises: Get the node data of the human joint points corresponding to the target interactive object in the extended real space.

3. The method according to claim 1, wherein the target human joint point comprises a plurality of human joint points, the preset interaction condition comprises a joint point distribution condition, and determining whether the preset interaction condition is currently satisfied based on the node data of the target human joint point comprises: Determine whether the distribution data of the plurality of human joints meets the joint distribution condition in the preset interaction condition.

4. The method according to claim 1, wherein the preset interaction condition includes an interaction time range condition, and determining whether the preset interaction condition is currently satisfied based on the node data of the target human joint point comprises: Determine whether the timestamp information of the node data of the target human joint point meets the interaction time range condition in the preset interaction condition.

5. The method according to claim 1, wherein the step of obtaining the node data of the target human joint point comprises: acquiring sensor data and / or image data from a target tracking device; determining tracking data of the target tracking device based on the sensor data and / or the image data; Based on the tracking data of the target tracking device, node data of the target human joint point is determined.

6. The method according to claim 5, wherein determining the node data of the target human joint based on the tracking data of the target tracking device comprises: The target algorithm model is used to determine the node data of the target human joint points based on the tracking data of the target tracking device; wherein the target algorithm model is trained using data matching the preset human skeleton model.

7. The method according to claim 5, wherein the target tracking device includes at least a head-mounted device, and the node data of the target human joint point is relative position node data determined based on a root node, and the root node is determined based on the tracking data of the head-mounted device.

8. The method according to claim 1, wherein the node data includes position information, the preset interaction condition includes preset interaction position information, and determining whether the preset interaction condition is currently satisfied based on the node data of the target human joint comprises: Determine whether the position information of the target human joint point matches the preset interaction position information in the preset interaction condition.

9. The method according to claim 8, wherein the node data further includes speed information, the preset interaction condition further includes preset speed information corresponding to the preset interaction position information, and determining whether the preset interaction condition is currently satisfied based on the node data of the target human joint comprises: Determine whether the speed information of the target human joint point matches the preset speed information in the preset interaction condition.

10. The method according to any one of claims 1 to 9, wherein the preset interaction condition further includes a preset interaction action condition, and before displaying the target virtual resource in the extended real space, the method further includes: Identify target interaction actions; Determine whether the target interactive action satisfies the preset interactive action condition.

11. The method according to claim 10, wherein the identifying target interaction action comprises: identifying the target interaction action according to the tracking data of the target tracking device and a first action recognition model; Alternatively, the target interaction action is identified based on the node data and a second action recognition model.

12. The method of claim 1, wherein the target tracking device comprises at least a lower limb tracking device.

13. The method according to claim 12, wherein the target tracking device further comprises a head-mounted device, or a head-mounted device and a handle device.

14. The method according to claim 12, wherein the target tracking device further comprises a head-mounted device, and the node data is determined based on tracking data of the target tracking device and hand tracking data, and the hand tracking data is obtained using the head-mounted device.

15. An interactive device based on extended reality, the device comprising: An acquisition module, configured to acquire node data of a target human joint point; wherein the target human joint point includes one or more human joint points in a preset human skeleton model, and the node data is determined based on tracking data of a target tracking device; The first determination module is used to display the target virtual resource in the extended real space when it is determined based on the node data of the target human joint point that a preset interaction condition is currently met.

16. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the augmented reality-based interaction method described in any one of claims 1-14.

17. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the interactive method based on extended reality according to any one of claims 1 to 14.

Images