Virtual object interaction control method and device

Abstract

The invention discloses a virtual object interaction control method and device, and belongs to the field of virtual reality. The virtual object interaction control method comprises the steps that first input of a first user to a first virtual character is received, and the first input is used for inputting first pose information; the first virtual character is constructed based on standard pose information corresponding to a plurality of joints of the first user under the condition that the first user executes the target standard action; in response to the first input, determining a first action instruction based on the first pose information; based on the first action instruction, a target object in the virtual environment is controlled to execute target pose information, and the target object comprises at least one of the first virtual character and a second object bound with the first virtual character. According to the virtual object interaction control method, the interaction feeling and immersion feeling of players can be improved.

Description

Technical Field

[0001] The present application belongs to the field of virtual reality, and in particular, relates to a method and device for interactive control of virtual objects. Background Art

[0002] With the development of technology, virtual reality (VR) technology is increasingly being used in people's daily lives. In related technologies, users in VR scenarios often wear motion capture devices to send instructions to virtual characters in the virtual environment to achieve interactive control between virtual and real life. However, VR applications currently on the market generally have problems such as insufficient interactivity, insufficient intelligent experience, and unnatural interaction with users, which affect the interactive experience and thus the user's sense of immersion and participation. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a virtual object interaction control method and device, which can realize accurate virtual reality interaction and improve the player's sense of interaction and immersion.

[0004] In a first aspect, the present application provides a virtual object interaction control method, the method comprising:

[0005] Receiving a first input from a first user to a first virtual character, the first input being used to input first posture information; the first virtual character is constructed based on standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action;

[0006] In response to the first input, determining a first action instruction based on the first posture information;

[0007] Based on the first action instruction, a target object in the virtual environment is controlled to execute target posture information, wherein the target object includes at least one of the first virtual character and a second object bound to the first virtual character.

[0008] According to the virtual object interaction control method of the present application, a first virtual character is constructed by constructing standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action, and the first virtual character is applied to a virtual reality scene for the user to perform virtual reality control. The virtual object can be precisely controlled in action through a small amount of sensor information, thereby achieving precise virtual reality interaction, which helps to improve the player's sense of interaction and immersion.

[0009] According to an embodiment of the present application, controlling the target object in the virtual environment to execute the target posture information based on the first action instruction includes:

[0010] In a case where a collision body corresponding to the first virtual character is determined from a plurality of objects in the virtual environment based on the first action instruction, the collision body is determined as the second object;

[0011] Determining a binding form between the second object and the first virtual character based on a collision type corresponding to the second object;

[0012] Binding the first virtual character and the second object based on the binding form;

[0013] Based on the first action instruction and the binding form, the first virtual character and the second object are controlled to execute the target posture information.

[0014] According to one embodiment of the present application, the binding form includes: a binding form for geometric offset based on a physical engine, a binding form for geometric offset based on scanning, a binding form for setting stiffness constraints and / or damping constraints, and at least one of a binding form for setting constant values.

[0015] According to an embodiment of the present application, controlling the target object in the virtual environment to execute the target posture information based on the first action instruction includes:

[0016] Based on the first action instruction, determining first posture information and sending the first posture information to a server;

[0017] Controlling the first virtual character to execute the first posture information;

[0018] Receiving second position information sent by the server, where the second position information is generated by the server after feasibility verification of the received first position information;

[0019] Based on the second posture information, correct the first posture information to obtain the target posture information;

[0020] Control the first virtual character to execute the target posture information.

[0021] According to an embodiment of the present application, the step of correcting the first posture information based on the second posture information to obtain the target posture information includes:

[0022] Based on the second posture information, a linear interpolation algorithm is used to correct the first posture information to obtain the target posture information.

[0023] According to an embodiment of the present application, when the server is connected to multiple clients, the method further includes:

[0024] The multiple clients respectively receive the second posture information sent by the server to control the movement of the first virtual character in each client based on the second posture information; the multiple clients include clients corresponding to the first user and clients corresponding to other users.

[0025] According to one embodiment of the present application, the target standard actions include: standing upright with both hands raised horizontally to the sides, standing upright with both hands raised, and standing upright with both hands hanging naturally.

[0026] In a second aspect, the present application provides a virtual object interaction control device, the device comprising:

[0027] A first processing module is configured to receive a first input from a first user to a first virtual character, wherein the first input is used to input first posture information; the first virtual character is constructed based on standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action;

[0028] A second processing module, configured to determine a first action instruction in response to the first input and based on the first posture information;

[0029] A third processing module is configured to control a target object in a virtual environment to execute target posture information based on the first action instruction, wherein the target object includes at least one of the first virtual character and a second object bound to the first virtual character.

[0030] According to the virtual object interaction control device of the present application, a first virtual character is constructed by constructing standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action, and the first virtual character is applied to a virtual reality scene for the user to perform virtual reality control. The virtual object can be precisely controlled in action through a small amount of sensor information, thereby achieving precise virtual reality interaction, which helps to improve the player's sense of interaction and immersion.

[0031] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the virtual object interaction control method as described in the first aspect above is implemented.

[0032] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the virtual object interaction control method as described in the first aspect above is implemented.

[0033] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the virtual object interaction control method as described in the first aspect above.

[0034] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0035] A first virtual character is constructed by constructing standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action, and the first virtual character is applied to a virtual reality scene for the user to perform virtual reality control. The virtual object can be precisely controlled in action through a small amount of sensor information, thereby achieving precise virtual reality interaction, which helps to improve the player's sense of interaction and immersion.

[0036] Furthermore, by determining the optimal binding form based on the collision type determined by the collision detection, and binding the first virtual character and the second object based on the determined binding form, a more realistic collision effect can be simulated, thereby improving the player's sense of reality and immersion in the virtual reality scene, thereby improving the user experience.

[0037] Furthermore, after the client receives the user's control instruction, the posture information of the virtual character is predicted based on the control instruction, so that the virtual character can be controlled before receiving the posture adjustment instruction sent by the server, which can ensure the smoothness of the virtual character's movements and thus enhance the player's immersion.

[0038] Furthermore, the first pose information is corrected by a linear interpolation algorithm so that it is gradually updated to the second pose information through a smooth trajectory, which can make the correction process of the virtual object smoother and more fluid, thereby improving the user's immersion and experience.

[0039] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0041] Figure 1 is a flowchart of a virtual object interaction control method provided in an embodiment of the present application;

[0042] Figure 2 is a schematic diagram of the principle of a virtual object interaction control method provided in an embodiment of the present application;

[0043] Figure 3is a structural diagram of a virtual object interaction control device provided in an embodiment of the present application;

[0044] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0046] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0047] The following, in conjunction with the accompanying drawings, describes in detail the virtual object interaction control method, virtual object interaction control device, electronic device and readable storage medium provided in the embodiments of the present application through specific embodiments and their application scenarios.

[0048] The virtual object interaction control method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0049] The terminal includes but is not limited to portable communication devices such as mobile phones or tablet computers. It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer.

[0050] The virtual object interaction control method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the virtual object interaction control method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The virtual object interaction control method provided in the embodiment of the present application is described below using an electronic device as an example of the execution subject.

[0051] like Figure 1 As shown, the virtual object interaction control method includes: step 110, step 120 and step 130.

[0052] It should be noted that in a VR scenario, one or more clients and a server electrically connected to each client may be involved.

[0053] For example, the multiple clients may include a first client corresponding to the first user and a second client corresponding to the second user, and the first client and the second client are electrically connected to the server end respectively.

[0054] During actual application, the first user sends an action instruction to the first client to control the first virtual character corresponding to the first user to perform a corresponding action, and the corresponding action performed by the first virtual character will be synchronized to the server; the second user sends an action instruction to the second client to control the second virtual character corresponding to the second user to perform a corresponding action, and the corresponding action performed by the second virtual character will also be synchronized to the server.

[0055] The following describes the method of this application using the client as the execution subject.

[0056] Step 110: receiving a first input from a first user to a first virtual character, where the first input is used to input first posture information; the first virtual character is constructed based on standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action;

[0057] In this step, the first user may be a player, an animal, or other object, which is not limited in this application.

[0058] The first posture information is information such as the real-time position posture of the first user.

[0059] During the actual implementation process, the first user's first posture information may be captured by a sensor or a motion capture device.

[0060] The first input is used to input the first position information.

[0061] The first input may be in the form of motion capture input or somatosensory gesture input, etc., which is not limited in this application.

[0062] The following description is made by taking the first user as a person as an example.

[0063] like Figure 2 As shown, in some embodiments, the first posture information is the current posture information corresponding to multiple joints of the first user, and the multiple joints may include: head joints, hand joints, corresponding arm joints, waist joints, corresponding leg joints and at least one of foot joints.

[0064] In this embodiment, the joints are skeletal points.

[0065] For example, during motion capture, a tracker is attached to a corresponding joint of a first user's body to capture first pose information.

[0066] In some embodiments, the number of multiple joints does not exceed ten.

[0067] Trackers can be attached to the player's head, two hands, two arms, waist, two knees, and two feet to capture standard posture information.

[0068] In the actual implementation process, an HTC headset can be used for the head, a VR handle can be used for the hands, and 7 infrared trackers can be set up accordingly for other joints of the body.

[0069] Each tracker will directly obtain the world transform information of each tracker in the virtual world through the SteamVR software and the OpenXR plug-in under Unreal, including the first-level pose information such as position, rotation and size.

[0070] The first virtual character is a virtual character corresponding to the first user in the virtual environment.

[0071] The first virtual character is constructed based on standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action.

[0072] In this step, the first virtual character may be a virtual character controlled by the first user in a VR scenario.

[0073] The first virtual character is consistent with or closer to the actual skeleton features of the first user, and has a posture model that conforms to the kinematic features of the first user.

[0074] The target standard action is a pre-designed action.

[0075] The number of target standard actions can be one or more.

[0076] In some embodiments, the target standard action may include at least two of: standing upright with both hands raised to the sides, standing upright with both hands raised, and standing upright with both hands hanging naturally.

[0077] In this embodiment, the target standard action is designed based on the category of the first user. When the category of the first user changes, the corresponding target standard action may also change accordingly. For example, the target standard actions corresponding to humans and animals may be different.

[0078] In some embodiments, the first virtual character is constructed based on standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action, which may include:

[0079] Acquire standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action;

[0080] Determining pre-calibration information based on standard pose information and standard information corresponding to the standard skeleton model;

[0081] The standard skeleton model is processed based on the pre-calibration information and the motion capture algorithm to construct a first virtual character corresponding to the first user.

[0082] In this embodiment, the standard pose information is the world transformation (WorldTransform) information of the tracker in the virtual world, including information such as position, rotation, and size.

[0083] The standard skeleton model is a general skeleton model used to characterize the category to which the first user belongs.

[0084] The standard skeleton model can be designed in advance, and in actual application, a matching standard skeleton model can be selected based on the first user.

[0085] The bone points in the standard bone model may include: pelvis, spine, left and right clavicles, left and right upper arms, left and right forearms, left and right hands, left and right rotation joints, neck, head, left and right thighs, left and right calves, left and right feet, and left and right rotation joints, etc.

[0086] The standard information is used to characterize the positional relationship between the joints (i.e., bone points) in the standard skeleton, the kinematic characteristics of each joint, and the distance between the joints.

[0087] The pre-calibration information is used to calibrate the standard skeleton model to obtain a skeleton model that conforms to the first user.

[0088] The pre-calibration information may be determined based on the difference between the standard pose information and the standard information corresponding to the standard skeleton model.

[0089] It is understandable that different first users may have different collected standard posture information, and the corresponding calibration information may also be different.

[0090] The motion capture algorithm may be any relevant algorithm, including but not limited to: a cyclic coordinate descent inverse dynamics (CCD IK) algorithm, a TwoBoneIK algorithm, a FABRIK algorithm, and a full body inverse kinematics (FullBodyIK) algorithm.

[0091] In the present application, the posture information of the first user is collected to infer the skeleton point position information of the first user, and then the skeleton point position information is input into the motion capture algorithm to obtain the target skeleton model, so as to complete the preliminary restoration of the posture of the first user.

[0092] For example, in the actual execution process, the first user can perform the above three postures in sequence to collect standard posture information through the tracker; then the standard skeleton model is processed based on the pre-calibration information and motion capture algorithm determined based on the collected standard posture information corresponding to the first user, and the target skeleton model corresponding to the first user (i.e., the first virtual character) can be obtained.

[0093] In this embodiment, by making the first user perform the target standard action to collect standard posture information, and then determining pre-calibration information based on the collected standard posture information and the standard information corresponding to the standard skeleton model, and constructing the first virtual character corresponding to the first user based on the pre-calibration information and the motion capture algorithm to process the standard skeleton model, it is possible to obtain the corresponding skeleton posture information in the virtual world in real time and accurately from a small amount of sensor information, which is suitable for VR and XR scenarios, and helps to improve the player's immersive experience.

[0094] According to the virtual object interaction control method provided in the embodiment of the present application, a first virtual character is constructed by constructing standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action, and the first virtual character is applied to a virtual reality scene for the user to perform virtual reality control. The virtual object can be precisely controlled in action through a small amount of sensor information, thereby achieving precise virtual reality interaction, which helps to improve the player's sense of interaction and immersion.

[0095] Step 120: In response to the first input, determine a first action instruction based on the first posture information;

[0096] In this step, the first action instruction is an instruction for controlling the first virtual character to perform a corresponding action, including but not limited to: pushing, pulling, walking, running and jumping.

[0097] For example, when the first user performs the action of "walking left" in the real world, the motion capture device collects the corresponding first pose information to generate an instruction for controlling the first virtual character to synchronously perform the action of "walking left" in the virtual environment based on the first pose information.

[0098] Step 130: Based on the first action instruction, control the target object in the virtual environment to execute the target posture information, the target object including at least one of the first virtual character and the second object bound to the first virtual character.

[0099] In this step, the second object may include other virtual characters or any virtual objects in the virtual environment, such as tables, chairs, doors, beds, and other virtual characters, etc., which are not limited in this application.

[0100] The target pose information is the updated pose information.

[0101] For example, when the first virtual character is at position P and the first user walks to the left, the first client receives the first action instruction "walk left", thereby controlling the first virtual character in the first client to walk to the left.

[0102] For another example, when the first user performs the action of "opening the door", the first client binds the door in the virtual environment to the first virtual character based on the received first action instruction, and controls the first virtual character to execute the target posture information of "opening the door".

[0103] In the actual execution process, collision detection can be performed first, and when a collision is detected, the colliding objects can be bound; when binding, the relative transformation Transform between the first virtual character and the second object can be directly bound to a fixed value, that is, no geometric offset is performed, which can be applied to non-entity fictional capture in some scenes, such as user interactive illuminants (UI) or static projections.

[0104] During the research and development process, the inventors discovered that in the related technologies, most virtual reality interactions only rely on VR handles to achieve interaction between players and virtual characters. This method is less natural and intuitive, affecting the user's sense of immersion and participation. In order to improve authenticity, in the related technologies, there is also a method of realizing interaction between virtual characters through motion capture. However, when performing motion capture, it is often necessary to bind a large number of sensors to the player to collect the player's posture information to improve the motion capture effect, which will also affect the player's sense of operation and experience.

[0105] In the present application, the first virtual character is constructed by constructing the standard posture information corresponding to multiple joints of the first user when the first user performs the target standard action. Only a small number of sensors need to be bound to accurately collect the player's posture information and construct an accurate virtual character, effectively improving the motion capture effect and the authenticity of the virtual character model.

[0106] In addition, the first virtual character constructed based on the above method is applied to a virtual reality scene for users to control virtual reality. Combining VR technology with general artificial intelligence can significantly improve the interaction experience between users and virtual environments, achieve a more intelligent and personalized virtual reality experience, and enable users to interact with the virtual environment in a natural and intuitive way, thereby enhancing users' sense of immersion and participation.

[0107] According to the virtual object interaction control method provided in the embodiment of the present application, a first virtual character is constructed by constructing standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action, and the first virtual character is applied to a virtual reality scene for the user to perform virtual reality control. The virtual object can be precisely controlled in action through a small amount of sensor information, thereby achieving precise virtual reality interaction, which helps to improve the player's sense of interaction and immersion.

[0108] The UE engine uses the built-in PhysX physics engine to simulate physical effects in the virtual environment. PhysX provides a wealth of physical properties, including rigid bodies, joints, and collision bodies, for simulating real object movement, collision, and grabbing and other VR interactions in the virtual environment. In some embodiments, when the first user's hand controller touches a virtual object, a physical joint constraint (Physics Constraint) can be added to bind the object to be grabbed and the user's hand controller based on the collision detection result of PhysX. This grabbing mechanism enables users to achieve highly realistic object grabbing and interaction based on their own hand movements.

[0109] The specific implementation process of this method is described below.

[0110] In some embodiments, step 130 may include:

[0111] In a case where a collision body corresponding to the first virtual character is determined from a plurality of objects in the virtual environment based on the first action instruction, the collision body is determined as a second object;

[0112] Determining a binding form between the second object and the first virtual character based on a collision type corresponding to the second object;

[0113] Binding the first virtual character and the second object based on the binding form;

[0114] Based on the first action instruction and the binding form, the first virtual character and the second object are controlled to execute the target posture information.

[0115] In this embodiment, the collision body may be a simple geometric shape (such as a sphere, a box, a capsule) or a complex geometric model (such as a convex polyhedron).

[0116] It can be understood that in UE, each object has one or more collision bodies for defining its collision shape.

[0117] The collision type may include, but is not limited to, information such as the physical material of the collision surface, the mass of the object, and damping.

[0118] Based on the collision type, you can determine the forces to apply, as well as modify the object's velocity and position, etc.

[0119] When performing collision detection, you can use the BVH algorithm to discretely screen collision bodies that may require fine detection, and then use precise detection algorithms such as the SAT (Separating Axis Theorem) or GJK (Gilbert-Johnson-Keerthi) algorithm to determine whether a collision occurs.

[0120] When it is determined that a collision occurs, the collision body is determined as the second object, and a real collision effect is simulated based on the collision type.

[0121] In the process of simulating a real collision effect, the binding form between the second object and the first virtual character may be determined based on the collision type.

[0122] It should be noted that, in this embodiment, different binding forms may be determined for different collision types to simulate different collision effects.

[0123] In some embodiments, the binding form may include: a binding form for geometric offset based on a physical engine, a binding form for geometric offset based on scanning, a binding form for setting stiffness constraints and / or damping constraints, and at least one of a binding form for setting constants.

[0124] In this embodiment, for objects of normal size or mass, a physical engine may be used to perform geometric offset to simulate real physical interaction effects, such as a hammer or a building block.

[0125] On this basis, stiffness and damping settings can be added to simulate some movements with large damping that need to reduce swing, such as holding chains, springs or rubber bands.

[0126] For objects that are large or small in weight, such as picking up a large object with both hands (because the object is too heavy to be moved by the Baby agent, the weight is set to infinity), you can use scanning to offset the geometry to push the physically simulated object.

[0127] For some imaginary grabs without entities in the scene, you can also directly bind the relative Transform to a fixed value without offsetting the geometry, such as user interaction illuminants (UI) or static projections.

[0128] In the actual implementation process, one of the above binding forms or a combination of the above binding forms can be used for binding to simulate a real collision effect.

[0129] According to the virtual object interaction control method provided in the embodiment of the present application, the optimal binding form is determined by the collision type determined by the collision detection, and the first virtual character and the second object are bound based on the determined binding form. This can simulate a more realistic collision effect, improve the player's sense of reality and immersion in the virtual reality scene, and thus improve the user experience.

[0130] In some embodiments, step 130 may include:

[0131] Based on the first action instruction, determining the first posture information and sending the first posture information to the server;

[0132] Controlling the first virtual character to execute the first posture information;

[0133] Receiving second posture information sent by the server, where the second posture information is generated by the server after feasibility verification of the received first posture information;

[0134] Based on the second posture information, the first posture information is corrected to obtain the target posture information;

[0135] The first virtual character is controlled to execute the target posture information.

[0136] In this embodiment, the second posture information is generated by the server after performing feasibility verification on different first posture information received from each client.

[0137] In some embodiments, feasibility verification may include checking whether it has crossed the boundary and whether it is legal.

[0138] It should be noted that the second pose information may be the same as the first pose information, or may be different from the first pose information.

[0139] The target pose information is the intermediate process target of each adjustment in the process of adjusting the virtual object from the first pose information to the second pose information.

[0140] When the first pose information is the same as the second pose information, the target pose information may also be the same as the first pose information and the second pose information.

[0141] For example, after receiving the first action instruction input by the first user to control the first virtual character to "walk right", the first client performs local prediction, simulates the movement of the first virtual character based on the first action instruction, and directly updates the local state without waiting for the server to respond, so as to control the first virtual character to walk right and keep the game smooth.

[0142] In addition, the first client sends the first posture information to the server, which receives and verifies the first posture information, such as checking whether it crosses the boundary and whether it is legal.

[0143] It can be understood that in some embodiments, when the server does not receive new motion control instructions sent by other clients, or the received motion control instructions do not conflict with the first pose information sent by the first client, the server determines that the first pose information is legal information. Then, the server determines the first pose information as the second pose information, applies the first pose information to the server status, and updates the position or rotation and other pose information corresponding to each object in the server.

[0144] In other embodiments, when the action control instructions sent by other clients received on the server side conflict with the first posture information sent by the first client, such as the first posture information is used to control the first virtual character to walk to the right, and there are other virtual characters (such as the second virtual character) walking to the left, the server side detects that the first virtual character and the second virtual character are about to collide, then the first posture information is considered illegal, and the posture information of the first virtual character needs to be updated based on the second posture information to apply the target posture information to the server-side state, and update the player's position and rotation and other states.

[0145] After the client corresponding to the first user receives the second pose information, if the second pose information is different from the first pose information (that is, it does not match the local prediction), the state of the current first virtual character can be corrected based on the received second pose information, rolled back to the state approved by the server, and any unsynchronized data can be repaired.

[0146] For the movement synchronization of VR characters, a prediction correction mechanism is used; each client will make predictive movements based on the received position information of other users to reduce the impact of network delay on movement.

[0147] At the same time, the server will regularly correct the positions of all users to ensure that the positions of all users in the same scene remain consistent, providing a consistent virtual environment experience.

[0148] According to the virtual object interaction control method provided in the embodiment of the present application, after the client receives the user's control instruction, the posture information of the virtual character is predicted based on the control instruction, so as to control the virtual character before receiving the posture adjustment instruction issued by the server, thereby improving responsiveness and smoothness, ensuring the smoothness of the virtual character's movements, and thus improving the player's sense of immersion.

[0149] In some embodiments, when the server is connected to multiple clients, the method may further include:

[0150] The multiple clients respectively receive the second posture information sent by the server to control the movement of the first virtual character in each client based on the second posture information; the multiple clients include clients corresponding to the first user and clients corresponding to other users.

[0151] In this embodiment, different clients may correspond to different users.

[0152] Each user controls the relevant virtual character in the client through the corresponding client.

[0153] For example, the second user may control the second virtual character corresponding to the second user through the client corresponding to the second user.

[0154] Among them, the second virtual character is constructed based on the standard posture information corresponding to multiple joints of the second user when the second user performs the target standard action. The specific construction method is similar to that of the first virtual character, and this application will not go into details here.

[0155] For example, after receiving the first action instruction input by the first user to control the first virtual character to "walk right", the first client performs local prediction, simulates the movement of the first virtual character based on the first action instruction, and directly updates the local state without waiting for the server to respond, so as to control the first virtual character to walk right and keep the game smooth.

[0156] In addition, the first client also sends the first posture information to the server, which receives and verifies the first posture information, such as checking whether it crosses the boundary and whether it is legal.

[0157] When the server considers that the first pose information is illegal, the pose information of the first virtual character is updated based on the second pose information to apply the target pose information to the server-side state, update the player's position and rotation and other states, and broadcast the second pose information to other clients connected to it for communication, so that other clients can update the position or rotation and other pose information corresponding to the relevant objects in the local end based on the received second pose information.

[0158] This application adopts an event-driven synchronization mechanism for VR input. When a user performs operations in a VR environment, such as moving or grabbing objects, these operations will be sent to each server in the form of events; after receiving the event, the server will broadcast it to the clients of all other users sharing the same area with the user to ensure that all users can see the user's operations synchronously.

[0159] According to the virtual object interaction control method provided in the embodiment of the present application, the server side simulates the motion trajectory of the object according to the physical engine and synchronizes the result to other clients, which can ensure that all users' interactions with virtual objects are based on the same physical simulation, achieving a real and natural object interaction experience, thereby improving the sense of immersion.

[0160] In some embodiments, based on the second posture information, correcting the first posture information to obtain the target posture information may include:

[0161] Based on the second pose information, the linear interpolation algorithm is used to correct the first pose information to obtain the target pose information.

[0162] In this embodiment, the linear interpolation algorithm is an interpolation method, which uses the function values ​​of several known points in a certain interval to make an appropriate specific function, and uses the specific function as the approximate value of the function at other points in the linear interpolation interval.

[0163] In the actual execution process, any achievable linear interpolation algorithm may be used for interpolation processing, which may be specifically selected based on the motion type, and this application does not limit this.

[0164] The following uses the movement type of walking, running and other translation types as an example to illustrate the implementation method of interpolation.

[0165] During the interpolation process, a dead reckoning algorithm (DR) may be used for processing.

[0166] Basic Newtonian kinematics gives the formula:

[0167]

[0168] Among them, Qt is the target position; P t is the position of the first virtual character on the client corresponding to the first user after the target duration; P′ t is the position of the first virtual character on the server after the target duration; V′0 is the speed predicted by the client (i.e., the first pose information); V0 is the speed sent by the server (i.e., the second pose information); T is the target duration; A′0 is the acceleration.

[0169] Based on the above formula, it can be known that, assuming that the second posture information sent by the server to the client includes: the first virtual object is currently at position P′0, and moves forward at a speed of V′0 and an acceleration of A′0; based on the above formula, it can be predicted that after the target time length T, the object on the server will be at position Qt.

[0170] On this basis, if the speed changes (in magnitude or direction) after the server notifies the client, the client will get an incorrect prediction result. The algorithm does not restart the prediction, but interpolates the object to gradually switch to the correct motion state along a smooth motion trajectory.

[0171] The specific mathematical process is as follows:

[0172]

[0173]

[0174]

[0175]

[0176] Among them, V b is the intermediate process speed (i.e. target posture information) during each correction process; P t is the position of the first virtual character on the client corresponding to the first user after the target duration; P′ t is the position of the first virtual character on the server after the target duration; V′0 is the speed before correction (i.e. the first pose information); V0 is the speed sent by the server (i.e. the second pose information); T is the target duration; Qt is the intermediate process position in each correction process (i.e. the target pose information).

[0177] In the actual correction process, the client's speed is first corrected by linear interpolation to obtain the intermediate process speed Vb;

[0178] Then, the above formula is used to calculate the client position Pt and the server position P′t after T seconds respectively;

[0179] Finally, linear interpolation is used to further correct the client's position and calculate an intermediate process position Qt that is closer to the server's predicted position.

[0180] In this application, after each client fails in prediction, a correction algorithm is used to make each prediction as close to the actual state of the server as possible, thereby ensuring that the client continuously corrects the prediction results along a smooth trajectory. The calculation is simple and convenient, and helps to improve the player's immersion.

[0181] According to the virtual object interaction control method provided in the embodiment of the present application, the first posture information is corrected by a linear interpolation algorithm so that it is gradually updated to the second posture information through a smooth trajectory, which can make the correction process of the virtual object smoother and more fluent, thereby improving the user's immersion and experience.

[0182] The virtual object interaction control method provided in the embodiment of the present application can be executed by a virtual object interaction control device. In the embodiment of the present application, the virtual object interaction control device executing the virtual object interaction control method is taken as an example to illustrate the virtual object interaction control device provided in the embodiment of the present application.

[0183] An embodiment of the present application also provides a virtual object interaction control device.

[0184] like Figure 3 As shown, the virtual object interaction control device includes: a first processing module 310, a second processing module 320 and a third processing module 330.

[0185] The first processing module 310 is used to receive a first input from a first user to a first virtual character, where the first input is used to input first posture information; the first virtual character is constructed based on standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action;

[0186] A second processing module 320 is used to determine a first action instruction based on the first posture information in response to the first input;

[0187] The third processing module 330 is used to control the target object in the virtual environment to execute the target posture information based on the first action instruction, and the target object includes at least one of the first virtual character and the second object bound to the first virtual character.

[0188] According to the virtual object interaction control device provided in the embodiment of the present application, a first virtual character is constructed by constructing standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action, and the first virtual character is applied to a virtual reality scene for the user to perform virtual reality control. The virtual object can be precisely controlled in action through a small amount of sensor information, thereby achieving precise virtual reality interaction, which helps to improve the player's sense of interaction and immersion.

[0189] In some embodiments, the third processing module 330 may also be used to:

[0190] In a case where a collision body corresponding to the first virtual character is determined from a plurality of objects in the virtual environment based on the first action instruction, the collision body is determined as a second object;

[0191] Determining a binding form between the second object and the first virtual character based on a collision type corresponding to the second object;

[0192] Binding the first virtual character and the second object based on the binding form;

[0193] Based on the first action instruction and the binding form, the first virtual character and the second object are controlled to execute the target posture information.

[0194] In some embodiments, the third processing module 330 may also be used to:

[0195] Based on the first action instruction, determining the first posture information and sending the first posture information to the server;

[0196] Controlling the first virtual character to execute the first posture information;

[0197] Receiving second posture information sent by the server, where the second posture information is generated by the server after feasibility verification of the received first posture information;

[0198] Based on the second posture information, the first posture information is corrected to obtain the target posture information;

[0199] The first virtual character is controlled to execute the target posture information.

[0200] In some embodiments, the third processing module 330 may also be used to:

[0201] Based on the second pose information, the linear interpolation algorithm is used to correct the first pose information to obtain the target pose information.

[0202] In some embodiments, when the server is connected to multiple clients, the device may further include a fourth processing module for:

[0203] The multiple clients are respectively enabled to receive the second position information sent by the server, so as to control the movement of the first virtual character in each client based on the second position information; the multiple clients include the client corresponding to the first user and the clients corresponding to other users.

[0204] The virtual object interactive control device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0205] The virtual object interaction control device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0206] The virtual object interaction control device provided in the embodiment of the present application can achieve Figure 1 to Figure 2 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0207] In some embodiments, Figure 4 As shown, an embodiment of the present application also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 401, each process of the above-mentioned virtual object interaction control method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0208] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0209] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned virtual object interaction control method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0210] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0211] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned virtual object interaction control method when executed by a processor.

[0212] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0213] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned virtual object interaction control method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0214] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0215] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0216] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0217] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0218] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0219] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A virtual object interactive control method, characterized in that: include: Receiving a first input from a first user to a first virtual character, wherein the first input is used to input first pose information; The first virtual character is constructed based on standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action; In response to the first input, determining a first action instruction based on the first posture information; Based on the first action instruction, a target object in the virtual environment is controlled to execute target posture information, wherein the target object includes at least one of the first virtual character and a second object bound to the first virtual character.

2. The virtual object interactive control method according to claim 1, characterized in that: The controlling the target object in the virtual environment to execute the target posture information based on the first action instruction includes: In a case where a collision body corresponding to the first virtual character is determined from a plurality of objects in the virtual environment based on the first action instruction, the collision body is determined as the second object; Determining a binding form between the second object and the first virtual character based on a collision type corresponding to the second object; Binding the first virtual character and the second object based on the binding form; Based on the first action instruction and the binding form, the first virtual character and the second object are controlled to execute the target posture information.

3. The virtual object interactive control method according to claim 2, characterized in that: The binding forms include: at least one of a binding form for geometric offset based on a physical engine, a binding form for geometric offset based on scanning, a binding form for setting stiffness constraints and / or damping constraints, and a binding form for setting a constant value.

4. The virtual object interactive control method according to any one of claims 1 to 3, characterized in that: The controlling the target object in the virtual environment to execute the target posture information based on the first action instruction includes: Based on the first action instruction, determining first posture information and sending the first posture information to a server; Controlling the first virtual character to execute the first posture information; Receiving second position information sent by the server, where the second position information is generated by the server after feasibility verification of the received first position information; Based on the second posture information, correct the first posture information to obtain the target posture information; Control the first virtual character to execute the target posture information.

5. The virtual object interactive control method according to claim 4, characterized in that: The step of correcting the first posture information based on the second posture information to obtain the target posture information includes: Based on the second posture information, a linear interpolation algorithm is used to correct the first posture information to obtain the target posture information.

6. The virtual object interactive control method according to claim 4, characterized in that: In the case where the server is connected to multiple clients, the method further includes: The multiple clients respectively receive the second posture information sent by the server to control the movement of the first virtual character in each client based on the second posture information; the multiple clients include clients corresponding to the first user and clients corresponding to other users.

7. The virtual object interactive control method according to any one of claims 1 to 3, characterized in that: The target standard movements include at least two of: standing upright with both hands raised horizontally to the sides, standing upright with both hands raised, and standing upright with both hands hanging naturally.

8. A virtual object interactive control device, characterized in that: include: A first processing module, configured to receive a first input from a first user to a first virtual character, wherein the first input is used to input first pose information; The first virtual character is constructed based on standard posture information corresponding to multiple joints of the first user when the first user performs a target standard action; A second processing module, configured to determine a first action instruction in response to the first input and based on the first posture information; A third processing module is configured to control a target object in a virtual environment to execute target posture information based on the first action instruction, wherein the target object includes at least one of the first virtual character and a second object bound to the first virtual character.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the virtual object interaction control method as described in any one of claims 1-7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the virtual object interaction control method as described in any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the virtual object interaction control method according to any one of claims 1 to 7 is implemented.

Images