Space object processing method and device, storage medium and program product

Abstract

The embodiment of the invention provides a space object processing method and device, a storage medium and a program product. In the object processing method, a first virtual object can be moved to a target placement surface in a target space in response to a placement operation on the virtual object in the target space. In the process that the virtual object moves towards the target placement surface, the virtual object can be adjusted to a target posture, and the target posture is the posture that the target adsorption surface is placed on the target placement surface. Furthermore, before the virtual object is in contact with the target adsorption surface, the posture of the virtual object is adjusted to the target posture that the target adsorption surface is placed on the target placement surface, so that the risk of spatial position conflict between the virtual object and the target placement surface can be reduced in the adsorption process of the target adsorption surface and the target placement surface; therefore, the mode penetrating phenomenon in the mixed reality interaction process is reduced to a certain extent, and the reality sense of object placement operation in the target space is improved.

Description

Technical Field

[0001] The present application relates to the field of mixed reality technology, and in particular to a method, device, storage medium and program product for processing space objects. Background Art

[0002] Mixed Reality (MR) is a technology that can integrate virtual objects with the real world and allow these virtual objects to interact with objects in the real world. In an MR scene, users can manipulate virtual objects and place them on a plane in the real world. In some MR implementations, the effect of placing virtual objects in real scenes is not ideal, resulting in poor realism. Therefore, a new solution is needed. Summary of the invention

[0003] Multiple aspects of the present application provide a spatial object processing method, device, storage medium and program product to improve the placement effect of virtual objects in real scenes.

[0004] An embodiment of the present application provides a spatial object processing method, including: in response to a placement operation of a first virtual object in a mixed reality space, moving the first virtual object toward a target placement surface in the mixed reality space, wherein the target placement surface corresponds to a target adsorption surface on the first virtual object; in the process of the first virtual object moving toward the target placement surface, adjusting the first virtual object to a first target posture, wherein the first target posture is a posture in which the target adsorption surface is placed on the target placement surface; under the first target posture, placing the first virtual object onto the target placement surface with the target adsorption surface.

[0005] An embodiment of the present application provides a method for processing spatial objects, including: in response to a placement operation of a first virtual object in a mixed reality space, moving the first virtual object toward a target placement surface in the mixed reality space, the target placement surface being detected by a detection ray of a target adsorption surface on the first virtual object, and the direction of the detection ray being adapted to a target posture of the first virtual object being placed on the target placement surface with the target adsorption surface; when the first virtual object moves to an adsorption position corresponding to the target placement surface, rotating the first virtual object to a first target posture, the first target posture being a posture of the target adsorption surface being placed on the target placement surface.

[0006] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to: execute the steps in the method provided in the embodiment of the present application.

[0007] The embodiment of the present application also provides a computer-readable storage medium storing a computer program, which can implement the steps in the method provided in the embodiment of the present application when the computer program is executed by a processor.

[0008] The embodiment of the present application also provides a computer program product, including: a computer program / instructions, which, when executed by a processor, can implement the steps in the method provided in the embodiment of the present application.

[0009] In an embodiment of the present application, in response to a placement operation of a first virtual object in a mixed reality space, the first virtual object is moved to a target placement surface in the mixed reality space. During the movement of the first virtual object to the target placement surface, the first virtual object can be adjusted to a first target posture. After the first virtual object is adjusted to the first target posture, the target adsorption surface can be placed on the target placement surface under the first target posture. Among them, the first target posture is a posture in which the target adsorption surface is placed on the target placement surface. Based on this implementation, before the first virtual object contacts the target adsorption surface, the posture of the first virtual object is adjusted to a posture in which the target adsorption surface is placed on the target placement surface. During the adsorption process of the target adsorption surface and the target placement surface, the risk of spatial position conflict between the first virtual object and the target placement surface can be reduced, thereby reducing the "mold penetration" phenomenon in the mixed reality interaction process to a certain extent, and enhancing the realism of the object placement operation in the mixed reality space. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0011] Figure 1 A schematic flow chart of a space object processing method provided for an exemplary embodiment of the present application;

[0012] Figure 2 A schematic diagram of a mixed reality space provided for an exemplary embodiment of the present application;

[0013] Figure 3 A schematic diagram of determining a target placement surface provided by an exemplary embodiment of the present application;

[0014] Figure 4a A schematic diagram of a detection ray on an adsorption surface provided for an exemplary embodiment of the present application;

[0015] Figure 4b A schematic diagram of a detection ray on an adsorption surface provided for another exemplary embodiment of the present application;

[0016] Figure 5aA schematic diagram of a first virtual object detecting a second virtual object provided by an exemplary embodiment of the present application;

[0017] Figure 5b A schematic diagram of stacking multiple virtual objects provided by an exemplary embodiment of the present application;

[0018] Figure 6 A schematic diagram of correcting a posture in a process of moving a first virtual object provided by an exemplary embodiment of the present application;

[0019] Figure 7 A schematic diagram showing a placement projection of a first virtual object provided by an exemplary embodiment of the present application;

[0020] Figure 8 A schematic diagram of a plurality of virtual objects rebounding after collision provided by an exemplary embodiment of the present application;

[0021] Fig. 9 A schematic flow chart of a space object processing method provided for another exemplary embodiment of the present application;

[0022] Fig.10 A schematic diagram of first moving and then rotating to achieve placement of a virtual object provided by an exemplary embodiment of the present application;

[0023] Fig.11 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0025] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the situation of including at least one.

[0026] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0027] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the product or system including the element.

[0028] In some MR implementations, the bottom of a virtual object can be placed on the ground in the real scene, but other surfaces of the virtual object, such as the side, back, or top, cannot be placed. It also does not support placing a surface of a virtual object on a vertical surface or ceiling other than the ground. In the process of placing a virtual object, the posture of the virtual object may not match the placement surface, or the user's placement operation may be unreasonable. In this case, the virtual object may "penetrate the model".

[0029] In view of the above technical problems, a solution is provided in some embodiments of the present application. The technical solutions provided by the embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0030] Figure 1 is a flow chart of a space object processing method provided by an exemplary embodiment of the present application, the method may include: Figure 1 Steps shown:

[0031] Step 101: In response to a placement operation of a first virtual object in a target space, the first virtual object is moved to a target placement surface in the target space, where the target placement surface corresponds to a target adsorption surface on the first virtual object.

[0032] Step 102: When the first virtual object moves toward the target placement surface, the first virtual object is adjusted to a first target posture, where the first target posture is a posture in which the target adsorption surface is placed on the target placement surface.

[0033] Step 103: Under the first target posture, place the first virtual object on the target placement surface with the target adsorption surface.

[0034] The execution subject of this embodiment may be a device with spatial computing capabilities, such as a virtual reality (VR) device, a mixed reality (MR) device, an augmented reality (AR) device, or other types of devices capable of spatial computing, which are not limited in this embodiment. The target space provided by the above-mentioned VR device, MR device, and AR device may be a VR space, an MR space, and an AR space, respectively.

[0035] The following will take the execution subject as an MR device as an example to exemplify the spatial object processing method provided in this embodiment. The target space displayed by the MR device can be a mixed reality space, which integrates the real space and the virtual space. Among them, the real space displayed by the MR device can be obtained by the MR device using an image sensor to collect images and model the real space. The MR device can use an image sensor (such as a depth camera, an infrared camera, etc.) to capture the image and depth information of the real space, and convert the data collected by the image sensor into a three-dimensional space model through image processing and computer vision technology. As the position of the user changes when wearing the MR device and the environment changes, the MR device can continuously update the three-dimensional space model so that the displayed real scene is aligned with the real space. The virtual scene in the mixed reality space can be obtained by three-dimensional modeling of the target space and objects in the target space, or it can be a virtual scene automatically generated by an algorithm, which is not limited in this embodiment. Figure 2 The mixed reality space is shown in Figure 2 In the mixed reality space, the ground in the real scene and virtual objects can be displayed, and virtual objects can be placed on the ground in the real scene to achieve virtual-reality interaction.

[0036] In step 101, the first virtual object can be any virtual object in the target space, such as virtual tables and chairs, virtual computers, virtual furniture / appliances, virtual decorative ornaments, etc., depending on the application scenario. For example, in some e-commerce application scenarios, when users order furniture products through an online platform, the MR device can display the user's corresponding home environment and the virtual objects corresponding to the furniture products in the target space. The user can place the virtual objects corresponding to the furniture products in the target space, thereby simulating the arrangement of the furniture products in the real space. For another example, in some teaching scenarios, when teachers are teaching, the MR device can display the classroom scene and virtual teaching aids in the target space. The teacher can place the virtual teaching aids on the blackboard or desktop to teach more intuitively.

[0037] In MR, the placement operation of virtual objects can be achieved through the "adsorption" property of virtual objects, which ensures that virtual objects can be accurately aligned and stably attached to the specific surface of the carrier object. In this embodiment, for the convenience of description and distinction, the surface of the virtual object used to be placed on the carrier object is described as the adsorption surface. The surface of the carrier object used to carry the virtual object is described as the placement surface. In this embodiment, the carrier object can be any object that can carry objects in the real space, such as the ground, wall, tabletop, ceiling, window sill, etc., or it can be a virtual ground, virtual wall, virtual desktop, virtual ceiling or other virtual object in the virtual space, and this embodiment does not limit it.

[0038] In step 101, the placement operation of the first virtual object in the target space may be initiated by the user or by the MR device itself, which is not limited in this embodiment. For example, in some scenarios, when the user displays virtual furniture in a home environment, the user may manually initiate the placement operation of the virtual furniture in the real space. For example, the MR device may determine the virtual furniture selected by the user in the target space by detecting the user's pinching gesture, and move the virtual furniture in the target space by detecting the user's moving gesture. When the placement condition is met between the target adsorption surface and the target placement surface of the virtual furniture, the MR device may place the virtual furniture on the target placement surface with the target adsorption surface according to the user's placement operation of the virtual furniture. In some scenarios, when the user moves the first virtual object, if the placement condition is met between the target adsorption surface and the target placement surface of the first virtual object, the MR device may output placement prompt information, such as a prompt information of "relax to complete the placement". If the user's release operation is detected, the MR device may be considered to have detected the user's placement operation on the first virtual object.

[0039] For another example, in some scenarios, when the MR device displays the placement effects of multiple virtual furniture to the user, it can identify multiple placement positions in the real space, and move multiple virtual furniture in the target space according to the mapping relationship between the virtual furniture and the placement position. When the placement conditions are met between the target adsorption surface and the target placement surface of the virtual furniture, the MR device can automatically trigger the operation of placing the virtual furniture on the target placement surface with the target adsorption surface. For example, the MR device can move the virtual TV in the target space, and when the placement conditions are met between the bottom surface of the virtual TV and the TV cabinet in the mixed reality space, the MR device can automatically trigger the placement operation of placing the bottom surface of the virtual TV on the TV cabinet. For another example, the MR device can move the virtual desk lamp in the target space, and when the placement conditions are met between the bottom surface of the virtual desk lamp and the desk in the mixed reality space, the MR device can automatically trigger the placement operation of placing the bottom surface of the virtual desk lamp on the desk, which will not be listed one by one.

[0040] Optionally, the placement condition between the target adsorption surface and the target placement surface of the virtual object may include: the distance between the target adsorption surface on the virtual object and the target placement surface is less than or equal to the set adsorption threshold. The adsorption threshold may be determined according to the type of the target placement surface and / or the type of the virtual object. The type of the target placement surface may include: at least one of the ground, ceiling, chair and desktop. The type of the virtual object may depend on the specific application scenario. For example, in some virtual home decoration scenarios, the type of virtual objects may include but is not limited to: virtual objects of different sizes such as air conditioners, electric fans, televisions, and rice cookers. The adsorption thresholds of different types of target placement surfaces are different. For example, the adsorption threshold of the ground may be 30 cm, and the adsorption threshold of the ceiling may be 20 cm. The adsorption thresholds of different types of virtual objects are also different. For example, the adsorption threshold of the air conditioner is 35 cm, the adsorption threshold of the television is 25 cm, and the adsorption threshold of the rice cooker is 10 cm. In some embodiments, for any adsorption surface, the adsorption threshold corresponding to the adsorption surface may be determined according to the type of the candidate target placement surface corresponding to the adsorption surface. In other embodiments, for any adsorption surface, the adsorption threshold corresponding to the adsorption surface may be determined according to the type of the adsorption surface. In some other embodiments, for any adsorption surface, a first adsorption threshold can be determined according to the type of the adsorption surface, and a second adsorption threshold can be determined according to the type of the candidate target placement surface corresponding to the adsorption surface. The adsorption threshold corresponding to the adsorption surface is determined based on the average of the first adsorption threshold and the second adsorption threshold.

[0041] Among them, the placement conditions between the target adsorption surface of the virtual object and the target placement surface may include: the type of the virtual object is adapted to the type of the placement surface, and the adaptation refers to the combination of the virtual object and the placement surface, which satisfies the functional attributes of the virtual object and the placement surface. For example, when the virtual air conditioner is a wall-mounted air conditioner, it is adapted to the wall-type placement surface; when the virtual air conditioner is a floor-standing air conditioner, it is adapted to the ground-type placement surface; when the virtual air conditioner is a central air conditioner, it is adapted to the ceiling-type placement surface.

[0042] Optionally, the target adsorption surface of the virtual object and the target placement surface satisfying the placement condition may include: the distance between the target adsorption surface on the virtual object and the target placement surface is less than or equal to a set adsorption threshold, and the type of the virtual object is compatible with the type of the placement surface. For example, when the first virtual object is a wall-mounted air conditioner, if the distance between the first virtual object and the wall is less than 35 cm, it can be determined that the first virtual object and the target placement surface satisfy the placement condition.

[0043] When the first virtual object includes one or more adsorption surfaces, the target adsorption surface of the first virtual object may be an adsorption surface specified on the first virtual object, or may be any adsorption surface detected in the target space suitable for placing the first virtual object. Any adsorption surface detected suitable for placing the first virtual object matches the placement intention corresponding to the placement operation of the first virtual object, and the placement intention may be determined by the distance between the target adsorption surface and different placement surfaces and / or the placement attribute of the first virtual object. Among them, the placement attribute of the first virtual object is used to describe the requirements and characteristics of the first virtual object when it is placed, and the requirements and characteristics are used to meet the better functions and use effects of the first virtual object. For example, the placement attribute of a vertical air conditioner is: the bottom is placed on the ground, the placement attribute of a wall-mounted air conditioner is: the back is placed on the wall, the placement attribute of a wall-mounted washing machine is: the back is placed on the wall, and the placement attribute of a desktop computer is: the bottom is placed on the desktop. In mixed reality, the placement surface corresponding to the target adsorption surface is described as the target placement surface, and the target placement surface may be an adsorption surface specified in the mixed reality space, or may be any placement surface detected in the mixed reality space suitable for adsorption with the target adsorption surface. Any placement surface can be the ground, wall, or surface of other virtual objects in the target space, and this embodiment does not limit this. The determination method of the target adsorption surface and the target placement surface will be introduced in subsequent embodiments and will not be repeated here.

[0044] The MR device may move the first virtual object toward the target placement surface in response to the placement operation on the first virtual object.

[0045] In step 102, during the process of the first virtual object moving toward the target placement surface, the MR device may adjust the posture of the first virtual object to the first target posture. In three-dimensional space, the posture of an object refers to the direction of the object in space, which can be described by the rotation angle of the object. Optionally, the rotation angle of the object can be represented by Euler angles, quaternions, rotation matrices or axis angles.

[0046] Among them, the first target posture is the posture of the first virtual object placed on the target placement surface with the target adsorption surface. When the first virtual object is placed on the target placement surface with the target adsorption surface, the relationship between the target adsorption surface and the target placement surface meets at least one of the requirements of physics, the functional requirements of the object, and the aesthetic requirements. For example, when the first virtual object is placed on the target placement surface with the target adsorption surface, the contact area between the target adsorption surface and the target placement surface is greater than the set area threshold, and the area threshold enables the target adsorption surface and the target placement surface to fit closely. Or, the relative angle between the target adsorption surface and the target placement surface is less than the set angle threshold, and the angle threshold enables the adsorption force between the target adsorption surface and the target placement surface to be approximately perpendicular to the contact surface between the target adsorption surface and the target placement surface, thereby increasing the adsorption force between the target adsorption surface and the target placement surface. Or, the placement attribute of the target adsorption surface is adapted to the type of the target placement surface, for example, a wall-mounted air conditioner is suitable for being placed on the wall with its back, and a vertical air conditioner is suitable for being placed on the ground with its bottom surface. When the first virtual object is in the first target posture, the target placement surface is adsorbed with the target adsorption surface, so that the first virtual object can be placed on the target placement surface according to the placement requirements, and remain stable and not fall off.

[0047] In some optional embodiments, the first target posture can be determined according to the posture of the target placement surface. For example, the posture of the target placement surface can be directly used as the first target posture. In other embodiments, a posture close to the posture of the target placement surface can be used as the first target posture, and this embodiment does not limit it. Among them, the posture close to the posture of the target placement surface may include: a posture whose angle difference of the rotation angle corresponding to the posture of the target placement surface is less than a set angle threshold, and the set angle threshold can make the contact area between the target adsorption surface and the target placement surface greater than the set area threshold, or, make the adsorption force between the target adsorption surface and the target placement surface can be approximately perpendicular to the contact surface between the target adsorption surface and the target placement surface to meet the physical requirements.

[0048] In the process of the first virtual object moving toward the target placement surface, the posture of the first virtual object can be dynamically adjusted so that the posture of the first virtual object keeps changing until the posture of the first virtual object is adjusted to the first target posture. When the posture of the first virtual object is dynamically adjusted, the first virtual object can be directly adjusted to the first target posture, or the posture of the first virtual object can be adjusted for multiple times in succession until it is adjusted to the first target posture. After the posture of the first virtual object is adjusted to the first target posture, in step 103, the first virtual object can be placed on the target placement surface with the target adsorption surface under the first target posture, thereby realizing the operation of placing the first virtual object on the target placement surface in the target space. Optionally, the first virtual object can be placed on the target placement surface with the target adsorption surface in the manner that: the first virtual object is adsorbed on the target placement surface with the target adsorption surface, or the first virtual object is rigidly connected to the target placement surface with the target adsorption surface, which is not limited in this embodiment.

[0049] In this embodiment, in response to the placement operation of the first virtual object in the target space, the first virtual object is moved to the target placement surface in the target space. In the process of the first virtual object moving to the target placement surface, the first virtual object can be adjusted to the first target posture. After the first virtual object is adjusted to the first target posture, the target adsorption surface can be placed on the target placement surface under the first target posture. Among them, the first target posture is a posture in which the target adsorption surface is placed on the target placement surface. Based on this implementation, before the first virtual object contacts the target adsorption surface, the posture of the first virtual object is adjusted to a posture in which the target adsorption surface is placed on the target placement surface. During the adsorption process of the target adsorption surface and the target placement surface, the risk of spatial position conflict between the first virtual object and the target placement surface can be reduced, thereby reducing the "mold penetration" phenomenon in the mixed reality interaction process to a certain extent, and enhancing the realism of the object placement operation in the target space.

[0050] In an embodiment of the present application, the MR device can detect the plane in the target space based on the plane detection capability to obtain the plane in the target space, such as a horizontal plane, a vertical plane, etc. These planes can be flat surfaces or approximately flat surfaces, which are not limited in this embodiment. Among them, the horizontal plane may include a horizontal ground and a ceiling. The vertical plane may include a vertical plane and a plane with a certain angle to the vertical direction, such as an inclined vertical plane. The MR device can detect and update the size, position and type of the plane in real time to construct all physical planes and virtual planes in the target space in real time. The MR device can select all or part of the planes from the detected planes as at least one placement surface.

[0051] In some optional embodiments, when the MR device selects a target adsorption surface from among the adsorption surfaces of the first virtual object, it may analyze at least one adsorption surface of the first virtual object. Optionally, the at least one adsorption surface of the first virtual object may include: at least one of the top surface, the bottom surface, and the side surfaces in different directions of the first virtual object. The MR device may select, from among the at least one placement surface, candidate placement surfaces corresponding to each of the at least one adsorption surface of the first virtual object.

[0052] Optionally, for any adsorption surface, the MR device may obtain the distance between the adsorption surface and the at least one placement surface, and select a placement surface that is closer or closest to the adsorption surface from the at least one placement surface as a candidate placement surface corresponding to the adsorption surface. The candidate placement surface for any adsorption surface may be one or more. Figure 3 As shown, one of the adsorption surfaces of the first virtual object may be the back side of the first virtual object, and the MR device may detect the distances between the back side and the placement surface a, the placement surface b, and the placement surface c respectively through multiple detection rays emitted from the back side. Figure 3 In the illustration, the placement surface a is closest to the back surface of the first virtual object, and the placement surface a can be used as the target placement surface.

[0053] After determining the candidate placement surface, the MR device can determine the target adsorption surface from the at least one adsorption surface according to the placement attribute of the first virtual object and / or the distance between the at least one adsorption surface and the corresponding candidate placement surface. The placement attribute of the first virtual object is used to describe the requirements and characteristics of the first virtual object when it is placed, and the requirements and characteristics are used to meet the optimal functions and usage effects of the first virtual object. For example, the placement attribute of a vertical air conditioner is: the bottom is placed on the ground, the placement attribute of a wall-mounted air conditioner is: the back is placed on the wall, the placement attribute of a wall-mounted washing machine is: the back is placed on the wall, and the placement attribute of a desktop computer is: the bottom is placed on the desktop.

[0054] In some optional embodiments, the MR device may determine the target adsorption surface from the at least one adsorption surface according to the placement properties of the first virtual object. The MR device may select an adsorption surface that matches the placement properties of the first virtual object from the at least one adsorption surface as the target adsorption surface. Optionally, the placement properties of the first virtual object may carry an adsorption surface identifier, and the MR device may determine the adsorption surface corresponding to the adsorption surface representation from the at least one adsorption surface as the target adsorption surface. For example, when the first virtual object is a virtual floor-standing air conditioner, the MR device may determine the bottom surface of the first virtual object as the target adsorption surface. For another example, when the first virtual object is a virtual wall-mounted air conditioner, the MR device may determine the back surface of the first virtual object as the target adsorption surface.

[0055] In other embodiments, the MR device may determine the target adsorption surface from the at least one adsorption surface according to the distance between the at least one adsorption surface and the respective corresponding candidate placement surfaces. Optionally, the MR device may select the adsorption surface corresponding to the minimum distance as the target adsorption surface according to the principle of proximity placement and the distance between the at least one adsorption surface and the respective candidate placement surfaces.

[0056] In some other optional embodiments, the MR device may preferentially select a target adsorption surface according to the placement attribute of the first virtual object, and if there is no suitable target adsorption surface, select a target adsorption surface according to the distance between the at least one adsorption surface and the corresponding candidate placement surface. In this embodiment, the MR device may determine whether there is an adsorption surface that matches the placement attribute of the first virtual object among the at least one adsorption surface; if so, the adsorption surface that matches the placement attribute of the first virtual object may be used as the target adsorption surface; if not, the MR device selects the adsorption surface corresponding to the minimum distance according to the distance between the at least one adsorption surface and the respective candidate placement surfaces as the target adsorption surface.

[0057] After determining the target adsorption surface based on the above-mentioned implementation, the MR device may determine the target placement surface according to the candidate placement surfaces corresponding to the target adsorption surface. Optionally, when the target adsorption surface corresponds to multiple candidate placement surfaces, the MR device may use the candidate placement surface closest to the target adsorption surface as the target placement surface. In addition, in some optional embodiments, when the target adsorption surface corresponds to multiple candidate placement surfaces, the MR device may use the candidate placement surface that is adapted to the placement attributes corresponding to the target adsorption surface as the target placement surface. For example, when the first virtual object is a virtual floor-standing air conditioner, if the candidate placement surfaces of the first virtual object include: the ground and the desktop, the ground can be selected as the target placement surface.

[0058] Based on the above implementation, the MR device can automatically and accurately determine the target adsorption surface and the target adsorption surface according to the distance between the adsorption surface and the placement surface and the placement attribute of the first virtual object, so as to accurately identify the placement intention for the first virtual object.

[0059] In some optional embodiments, when MR selects the candidate placement surface corresponding to each of the at least one adsorption surface, it may use the detection ray on the at least one adsorption surface to detect at least one placement surface in the target space, and select a candidate placement surface for the at least one adsorption surface according to the result returned by the detection ray of each of the at least one adsorption surface. The following will be exemplified by taking any adsorption surface as an example.

[0060] In this embodiment, the top surface of the first virtual object can emit a detection ray to the ceiling in the target space, the bottom surface of the first virtual object can emit a detection ray to the ground in the target space, and any side surface of the first virtual object can emit a detection ray to any vertical surface in the target space. Assume that the target space contains k virtual objects, each virtual object includes a top surface, a bottom surface and m side surfaces, and the target space includes the ground (horizontal plane), the ceiling and n vertical surfaces. The top surface of the virtual object can be provided with a detection ray for detecting the ceiling, the bottom surface of the virtual object can be provided with a detection ray for detecting the ground, and the m side surfaces of the virtual object are provided with m*n detection rays for detecting the n vertical surfaces. The number of detection rays of k virtual objects for the horizontal ground and ceiling is k*2, and the number of detection rays of k virtual objects for n vertical surfaces is k*m*n. Of course, in some optional embodiments, any adsorption surface on the first virtual object can emit a detection ray to any plane in the target space to obtain k*(m+2)*(n+2) detection rays, which is not limited in this embodiment.

[0061] For any first adsorption surface among the at least one adsorption surface, the MR device can determine the detection ray of the first adsorption surface to the at least one placement surface, and use the detection ray of the first adsorption surface to the at least one placement surface to detect the at least one placement surface, obtain the distance between the first adsorption surface and the at least one placement surface, and determine the candidate placement surface corresponding to the first adsorption surface from the at least one placement surface according to the distance between the first adsorption surface and the at least one placement surface. In this embodiment, the direction of the detection ray of the first adsorption surface to any placement surface does not change with the posture change of the first virtual object. Furthermore, even if the posture of the first virtual object changes during the movement, the direction of the detection ray on the first adsorption surface to any placement surface remains unchanged, reducing the influence of the posture change of the first object on the distance detected by the detection ray, so as to more accurately detect the distance between the first adsorption surface and the at least one placement surface.

[0062] The following will take any placement surface in the at least one placement surface as an example to exemplify the optional implementation of determining the detection ray of the first adsorption surface to any placement surface. For ease of description, any placement surface in the at least one placement surface is described as the first placement surface.

[0063] In some optional embodiments, the MR device can determine the second target posture of the first virtual object placed on the first placement surface with the first adsorption surface according to the posture of the first placement surface. Optionally, the MR can directly use the posture of the first placement surface as the second target posture, or can use the posture whose deviation value from the posture of the first placement surface in each dimension is less than a certain angle threshold as the second target posture, which is not limited in this embodiment. The MR device can obtain the posture frame corresponding to the first virtual object under the second target posture. In this embodiment, the posture frame is the mapping result of mapping the first virtual object to the second target posture, that is, the posture frame is the final state when the first virtual object is placed on the first placement surface with the first adsorption surface. The posture frame of the first virtual object has the same shape as the first virtual object, and the posture frame does not change with the posture of the first virtual object. The posture frame of the first virtual object includes a final state adsorption surface corresponding to the first adsorption surface, and the posture of the final state adsorption surface is the posture of the first adsorption surface when the first virtual object is in the second target posture. The MR device can use the vertical ray from the point on the final state adsorption surface to the first placement surface as the detection ray of the first adsorption surface to the first placement surface. The following will provide an exemplary description of the configuration of the detection rays of any adsorption surface to any placement surface in conjunction with the accompanying drawings.

[0064] like Figure 4a As shown, taking the right side surface of the first virtual object as an example, when the first virtual object is adsorbed on the placement surface with its right side surface, the posture of the first virtual object should make its right side surface parallel to the placement surface to meet the adsorption requirements. In one embodiment, the posture of the placement surface can be used as the target posture of the right side surface of the first virtual object, and the second target posture of the first virtual object can be determined based on the target posture of the right side surface. As shown in Figure 4, a posture box corresponding to the first virtual object can be drawn based on the second posture, and in the posture box, the right side surface is mapped to the final adsorption surface under the second target posture. A vertical ray perpendicular to the placement surface is drawn from the point on the final adsorption surface, and a detection ray facing the placement surface is obtained.

[0065] Continue to refer Figure 4a As an example, taking the back side of the first virtual object as an example, when the back side of the first virtual object is adsorbed on the placement surface, the posture of the first virtual object should make its back side parallel to the placement surface to meet the adsorption requirement. In one embodiment, the posture of the placement surface can be used as the target posture of the back side of the first virtual object, and the second target posture of the first virtual object is determined according to the target posture of the back side. Figure 4a As shown, a posture frame corresponding to the first virtual object can be drawn according to the second target posture, in which the back side is mapped to the final state adsorption surface under the second target posture. A vertical ray perpendicular to the placement surface is drawn from the point on the final state adsorption surface to obtain the detection ray of the back side to the placement surface.

[0066] Continue to refer Figure 4b As shown in the figure, when the top surface of the first virtual object is placed on the ceiling as the adsorption surface, the posture of the ceiling can be used as the target posture of the top surface of the first virtual object, and the second target posture of the first virtual object is determined according to the target posture of the top surface. When the bottom surface of the first virtual object is placed on the horizontal plane as the adsorption surface, the posture of the horizontal plane can be used as the target posture of the bottom surface of the first virtual object, and the second target posture of the first virtual object is determined according to the target posture of the bottom surface. Figure 4b As shown, a posture frame corresponding to the first virtual object can be drawn according to the second target posture, in which the top surface S1 is mapped to a final adsorption surface parallel to the ceiling, and the bottom surface S2 is mapped to a final adsorption surface parallel to the horizontal plane. A vertical ray perpendicular to the ceiling is drawn from a point on the final adsorption surface corresponding to the top surface S1, and a detection ray from the top surface to the ceiling is obtained. A vertical ray perpendicular to the horizontal plane is drawn from a point on the final adsorption surface corresponding to the bottom surface S2, and a detection ray from the bottom surface S2 to the horizontal plane is obtained.

[0067] Based on this implementation, when the posture of the final adsorption surface remains unchanged, the posture of the detection ray of the first adsorption surface also remains unchanged, so that the distance between the first adsorption surface and the first placement surface can be detected more accurately, reducing the impact of the posture change of the first virtual object on the distance detection operation.

[0068] In some optional embodiments, at least one placement surface in the target space may include: a virtual placement surface on the second virtual object displayed in the target space. For example, the second virtual object may be a virtual table, a virtual operating table, a virtual shelf, etc. In this embodiment, a collision body is provided outside the second virtual object, and the collision body refers to a geometric shape or model used to detect the collision between the second virtual object and the environment or other virtual objects. The collision body of the second virtual object may be consistent with the size of the bounding box of the second virtual object. When the MR device detects the at least one placement surface, it may use the detection ray of the first adsorption surface on the virtual placement surface of the second virtual object to perform collision detection on the collision body of the virtual placement surface, and obtain the distance between the first adsorption surface and the virtual placement surface. In some embodiments, if the first adsorption surface is the target adsorption surface and the virtual placement surface on the second virtual object is the target placement surface, the MR device may adsorb the first adsorption surface on the virtual placement surface to stack the first virtual object on the second virtual object. Thus, the stacking effect of multiple virtual objects is achieved in the target space.

[0069] like Figure 5a As shown, the second virtual object can be implemented as a virtual washing machine in the target space, and a collision body is arranged outside the virtual washing machine. The first virtual object can perform collision detection on the collision body of the virtual washing machine through the detection ray on its bottom surface. Figure 5bAs shown, in the placement operation, the first virtual object can be placed with its bottom surface on the virtual plane of the virtual washing machine.

[0070] In some optional embodiments, when the MR device adjusts the first virtual object to the first target posture during the process of the first virtual object moving toward the target placement surface, the MR device may determine, according to the posture of the target placement surface, the posture of the first virtual object placed on the target placement surface with the target adsorption surface as the first target posture. Optionally, the MR device may directly use the posture of the target placement surface as the first target posture.

[0071] Optionally, during the process of the first virtual object moving toward the target placement surface, the MR device may directly adjust the first virtual object to the first target posture, and perform the movement operation of the first virtual object after the adjustment.

[0072] Optionally, during the movement of the first virtual object toward the target placement surface, the MR device may determine the real-time posture of the first virtual object at a plurality of consecutive moving moments according to the first target posture and the placement progress of the first virtual object, and dynamically and continuously adjust the first virtual object according to the real-time posture at the plurality of consecutive moments until the first virtual object is adjusted to the first target posture. Optionally, the plurality of consecutive moments may be determined according to the screen refresh rate of the MR device when displaying the target space. For example, when the MR device refreshes the image 60 times per second, there are 60 moving moments per second.

[0073] Optionally, the placement progress of the first virtual object can be determined according to the placement duration of the first virtual object. For example, in an automatic placement scenario, the placement operation of the first virtual object has a relatively fixed total execution time, and MR can determine the placement progress of the first virtual object according to the time the placement operation of the first virtual object has been executed. Optionally, the placement progress of the first virtual object can be determined according to the distance between the first virtual object and the target placement surface. The MR device can determine the posture of the first virtual object before the placement operation as the initial posture; for any moving moment in the continuous multiple moving moments, the MR device can determine the placement progress of the first virtual object at the moving moment according to the first target posture and the distance between the first virtual object and the target placement surface at the moving moment. The MR device can determine the posture change amount corresponding to the initial posture at the moving moment according to the placement progress, and update the initial posture according to the posture change amount to obtain the real-time posture of the first virtual object at the moving moment.

[0074] Taking the moving time t as an example, the real-time posture Rt at time t can be expressed by the following formula: Rt=R0+(R-R0)*(1-Dt / D0), where R0 is the initial posture, R is the first target posture, Dt is the distance of the first virtual object relative to the target placement surface at time t, and D0 is the distance of the first virtual object relative to the target placement surface before the placement operation, and D0 corresponds to R0.

[0075] like Figure 6 As shown, the target placement surface is in a vertical posture. Before the placement operation is initiated, the initial posture of the first virtual object is a horizontal posture. When the placement operation for the first virtual object is detected, the first virtual object can be mapped to a vertical posture to obtain Figure 6 The back of the first virtual object is mapped to the final adsorption surface on the attitude frame, and the vertical ray emitted from the final adsorption surface to the placement surface is the detection ray of the back of the first virtual object to the placement surface. If the target adsorption surface on the first virtual object is the back, in the process of moving the first virtual object to the placement surface, the first virtual object can be dynamically adjusted to a vertical posture, so that the back of the first virtual object and the target placement surface can fit tightly without penetrating the mold. Figure 6 As shown, at each moving time t, the initial posture can be updated according to the initial posture, the first target posture, the placement distance at time t, and the total placement distance to obtain the real-time posture at time t.

[0076] Based on this implementation, the MR device can calculate the real-time posture of the first virtual object at each movement moment of the first virtual object, and gradually adjust the posture of the first virtual object as the first virtual object moves toward the target placement surface. Thus, the effect of the first virtual object moving toward the target placement surface while continuously rotating can be presented.

[0077] It is worth noting that, in some embodiments, the placement operation of the first virtual object in the target space may be performed by the user, for example, the user may initiate a selection operation of the first virtual object in the real scene, and drag / move the first virtual object. When the distance between the target adsorption surface and the target placement surface of the first virtual object is less than the set distance threshold, the MR device may determine that the first virtual object meets the placement condition, and the automatic placement can be completed within a certain time range without the user's dragging / moving operation. The time range may be 2 seconds, 3 seconds or other possible durations. In this scenario, the MR device may output a placement release prompt message, for example, the release prompt message may be a message of "release to complete placement". After detecting that the user has let go, the MR device may perform the placement operation of the first virtual object within the set time range, and during the placement process, according to the initial posture of the first virtual object, the first target posture, the placement distance at each moving moment, and the total placement distance, the initial posture is updated to obtain the real-time posture at each moving moment, and the posture of the first virtual object is adjusted according to the real-time posture at each moving moment. Furthermore, during the automatic placement period after detecting that the user has let go, the MR device can also continuously perform posture correction.

[0078] In some optional embodiments, during the movement of the first virtual object to the target placement surface, the MR device may further determine the placement progress of the first virtual object according to the distance between the first virtual object and the target placement surface, and display the placement projection of the first virtual object in the target space according to the placement progress of the first virtual object. Among them, the optional implementation method of the MR device determining the distance between the first virtual object and the target placement surface can refer to the records of the aforementioned embodiments, which will not be repeated here. The placement projection of the first virtual object is used to project the outline or shadow of the first virtual object into the real environment to intuitively provide visual feedback on the placement progress of the first virtual object. If the user manually performs the placement operation of the first virtual object, during the placement process, the user can observe the preview effect of the first virtual object at the placement position according to the placement projection of the first virtual object, so as to better adjust its position and posture. The placement projection can be updated in real time according to the movement of the first virtual object. In this embodiment, a projection component can be configured for the first virtual object, and the projection component can be a two-dimensional rendering component, a three-dimensional rendering component or a video playback component, which is not limited in this embodiment. The MR device can provide the placement progress of the first virtual object to the projection component, and the projection component can display different placement projections according to the placement progress. For example, the placement progress can be a progress value of 0-1, and the projection component can display different projection effects according to different progress values ​​input. The projection effects corresponding to different progress values ​​can be gradient rings, and the size of the ring can be determined according to the size of the circumscribed circle of the target adsorption surface.

[0079] like Figure 7As shown, when the virtual camera is placed on the bottom surface of the real scene, the MR device can display a circular placement projection below the virtual camera. As the placement progress of the virtual camera changes, the transparency of the placement projection will gradually change, and the transparency is used to intuitively display the placement progress of the virtual camera.

[0080] In some embodiments, the placement operation of the first virtual object in the target space can be performed by the user. For example, the user can initiate a selection operation on the first virtual object in the real scene and drag / move the first virtual object. When the distance between the target adsorption surface and the target placement surface of the first virtual object is less than a set distance threshold, the MR device can determine that the first virtual object meets the placement conditions. At this time, the automatic placement can be completed within a certain time range without the user's dragging / moving operation. The time range can be 2 seconds, 3 seconds or other possible durations. In this scenario, the MR device can output a placement and release prompt information. For example, the release prompt information can be as follows: Figure 7 The message "Let go to complete placement" is shown. After detecting that the user has let go, the MR device can perform the placement operation of the first virtual object within a set time range, and during the placement process, according to the placement progress of the first virtual object, display the placement projection of the first virtual object in the target space. Therefore, during the automatic placement after detecting that the user has let go, the MR device can also continuously and consistently display the placement projection, so that manual placement and automatic placement can have a smooth transition.

[0081] In some optional embodiments, when the user loads multiple virtual objects in the target space, the MR device may output prompt information for staggering the positions of the multiple virtual objects. When the spatial positions of any two virtual objects in the target space conflict, the MR device may output prompt information to prompt the user to move any virtual object so that any single virtual object has enough display space.

[0082] In some optional embodiments, when the target space includes multiple virtual objects, position conflicts may occur among the multiple virtual objects during movement. When position conflicts occur among the multiple virtual objects, the MR device may adjust the positions of the multiple virtual objects to optimize the display effect.

[0083] Optionally, the MR device may display a third virtual object in the target space, and a collision body is arranged outside the third virtual object. When a collision operation between the first virtual object and the third virtual object is detected, the MR device may obtain the first position of the first virtual object when the collision occurs; after the collision operation, if the first virtual object is detected to move toward the inner space of the third virtual object, the MR device may obtain the second position of the first virtual object when the first virtual object stops moving. The MR device may determine a rebound path from the second position to a specific position before the collision operation based on a ray pointing from the second position to the first position, wherein the specific position before the collision operation may be the first position, or a position located a certain distance before the first position, which is not limited in this embodiment. The MR device may display the moving state of the first virtual object in the target space based on the rebound path.

[0084] like Figure 8 As shown, after the MR device loads the first virtual object and the third virtual object in the target space, the first virtual object can be moved according to the movement or placement operation of the first virtual object. If a collision is detected between the first virtual object and the third virtual object, the first position of the first virtual object when the collision occurs is recorded, such as Figure 8 If it is detected that the first virtual object moves toward the inside of the third virtual object, the second position of the first virtual object can be obtained when the first virtual object stops moving, such as Figure 8 The MR device may draw a ray pointing from the P2 position to the P1 position, and determine the rebound path according to the ray. In some embodiments, the MR device may directly use the P1 position as the position after the first virtual object rebounds, and use the path pointing from the P2 position to the P1 position as the rebound path. In other embodiments, such as Figure 8 As shown, the MR device may select the position P1' before the position P1 as the position of the first virtual object after rebounding, and use the path from the position P2 to the position P1' as the rebound path.

[0085] Based on the above implementation, in a virtual reality scene, after any two virtual objects collide, the virtual object that actively collides can rebound along the moving path, reducing the risk of overlap and penetration between multiple virtual objects and improving the realism of mixed reality. Furthermore, when the virtual object that actively collides rebounds along the moving path, it can rebound a certain distance outward based on the position at the time of the collision, thereby further ensuring that there will be no conflict between different virtual objects.

[0086] The following will be combined with the attached Fig. 9 , taking the target space as a mixed reality space as an example, the spatial object processing method provided in the application embodiment is further exemplified.

[0087] like Fig. 9 As shown, after placing virtual object A in the mixed reality space, the MR device can move virtual object A in the mixed reality space according to the user's movement operation on virtual object A. When the user places virtual object B in the mixed reality space, the MR device can detect whether virtual object A and virtual object B collide. If a collision occurs, the MR device can prompt the user to move virtual object A to reduce spatial conflict. If virtual object A and virtual object B do not collide, the MR device can perform spatial calculations based on the detection rays of virtual object A and virtual object B.

[0088] like Fig. 9 As shown, the MR device can detect planes in the mixed reality space and construct spatial planes. The planes in the mixed reality space may include horizontal planes, vertical planes, etc. The detection rays on each adsorption surface of any virtual object can intersect with the planes in the mixed reality space respectively, and each detection ray can return an intersection hit result. For any detection ray, when it intersects with multiple planes, the first intersecting plane can be returned. If the mixed reality space contains a top surface, a bottom surface, and n vertical planes, taking virtual object A as an example, assuming that virtual object A contains a top surface, a bottom surface, and four side surfaces, the detection rays of virtual object A include: detection rays on the top surface corresponding to the ceiling, detection rays on the bottom surface corresponding to the horizontal ground, and 4*n detection rays corresponding to the n vertical planes can be emitted on the four side surfaces. The number of detection rays of virtual object A is 2+4*n. 2+4*n detection rays can return 2+4*n detection results. The MR device can identify empty results that do not generate any intersections in the detection results and filter the empty results. As shown Fig. 9 As shown, during the movement of the virtual objects, the operation of performing spatial calculations based on the detection rays of the virtual objects A and B is executed in multiple cycles.

[0089] like Fig. 9 As shown, after each acquisition of the filtered detection results, it can be determined whether the detected placement surface includes a locked placement surface, which refers to the placement surface screened out in the previous round of detection. If a locked placement surface is included, the MR device can screen the placement surfaces that meet the placement conditions from the locked placement surfaces, and select the target placement surface from the screened placement surfaces. The placement conditions include an adsorption threshold, which is a threshold used to trigger a placement operation. In the mixed reality space, when the distance between the adsorption surface of the virtual object and the placement surface is less than or equal to the adsorption threshold, the placement operation of the virtual object can be triggered.

[0090] like Fig. 9As shown, if the detection result does not include a locked placement surface, the MR device can clear the existing locked placement surface and select a new locked placement surface. In the process of selecting a new locked placement surface, the MR device can select a candidate placement surface for each adsorption surface on the virtual object based on the detection result. For example, among the multiple placement surfaces corresponding to any adsorption surface, the placement surface with the shortest vertical distance to the adsorption surface can be selected as the candidate placement surface. After determining the candidate placement surface for each adsorption surface, the placement surfaces that meet the placement conditions can be screened, and the target placement surface can be selected from the screened placement surfaces.

[0091] When selecting the target placement surface from the filtered placement surfaces, the target adsorption surface can be filtered out from the adsorption surfaces corresponding to the filtered placement surfaces according to the principle of placement attribute priority, and the candidate placement surface corresponding to the target adsorption surface can be used as the target placement surface. For example, if the filtered placement surfaces that meet the adsorption threshold are the ground and the wall, and the virtual air conditioner is a wall-mounted air conditioner, and the virtual air conditioner includes two adsorption surfaces, the bottom and the back, the back can be selected as the target adsorption surface, and the wall can be selected as the target placement surface. If there is no adsorption surface that matches the placement attribute of the virtual object among the adsorption surfaces corresponding to the filtered placement surfaces, the adsorption surface with the smallest distance to the corresponding candidate placement surface can be selected as the target adsorption surface.

[0092] After the target placement surface is screened out based on the above steps, the target placement surface can be locked and the placement operation of the virtual object can be triggered. During the placement operation, the virtual object can be moved closer to the target placement surface. During the movement of the virtual object, the virtual object can be dynamically corrected in posture, and the progress of the virtual object's dynamic placement can be displayed until the target placement surface is reached. During the movement of the virtual object to the target placement surface, if it is detected that the virtual object is released, the posture of the virtual object will continue to be corrected, and the progress of the virtual object's dynamic placement will be displayed until the target placement surface is reached.

[0093] like Fig. 9 As shown in the figure, during the process of moving a virtual object, if it is detected that the virtual object is interpenetrating with other virtual objects, the virtual object will be rebounded to the position before the collision when the virtual object is released. After rebounding, if the virtual object meets the placement conditions, the posture of the virtual object will continue to be corrected and the dynamic placement progress will be displayed until the target placement surface is reached.

[0094] In the above process, when the virtual object is dynamically corrected, the initial position of the virtual object before the placement operation can be obtained, and the total placement distance of the virtual object from the target placement surface at the beginning of the placement operation can be obtained. At each timestamp in the placement process, the current placement distance can be obtained by detecting the ray, and the placement progress of the virtual object can be determined based on the current placement distance and the total placement distance. According to the placement progress, the initial position can be corrected to obtain the current posture corresponding to each timestamp. Furthermore, the virtual object can be dynamically and continuously adjusted during the placement process so that the virtual object is getting closer and closer to the target placement posture, reducing the risk of penetration and collision between the virtual object and the target placement surface during placement.

[0095] In addition to the aforementioned embodiments, the embodiments of the present application also provide the following spatial object processing method, in which the MR device can respond to the placement operation of the first virtual object in the target space and move the first virtual object to the target placement surface in the target space. When the first virtual object moves to the adsorption position corresponding to the target placement surface, the first virtual object is rotated to the first target posture, and the first target posture is the posture of placing the target adsorption surface on the target placement surface. Among them, the target placement surface is detected by the detection ray on the target adsorption surface, and the direction of the detection ray is adapted to the target posture of the first virtual object placed on the target placement surface with the target adsorption surface. For details, please refer to the aforementioned embodiments and the records of the accompanying drawings, which will not be described again here. Optionally, when the direction of the detection ray on the target adsorption surface is adapted to the target posture, the detection ray can be perpendicular to the target placement surface and does not change with the change of the posture of the first virtual object, so that the distance between the target adsorption surface and the target placement surface can be detected more accurately, and the placement operation of the first virtual object can be triggered according to the distance.

[0096] The adsorption position corresponding to the target placement surface is used to align with the center point of the first virtual object. When the center point of the first virtual object moves to the target position, it can be determined that the first virtual object has moved to the adsorption position. The adsorption position corresponding to the target placement surface can be determined based on the center point of the first virtual object and the distance between the center point and the target adsorption surface. After determining the distance between the center point of the first virtual object and the target adsorption surface, the position point on the front side of the target placement surface corresponding to the distance can be used as the adsorption position. Then, if Fig.10 As shown, when the center point of the first virtual object moves to the target position, the first virtual object is rotated to the first target posture, so that the target adsorption surface can be adsorbed with the target placement surface.

[0097] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 to 104 can be device A; for another example, the execution subject of steps 101 and 102 can be device A, and the execution subject of step 103 can be device B; and so on.

[0098] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0099] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0100] Fig.11 A schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application is shown. Fig.11 As shown, the electronic device includes: a memory 1101 and a processor 1102 .

[0101] The memory 1101 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application program or method operating on the electronic device.

[0102] The processor 1102 is coupled to the memory 1101 and is used to execute a computer program in the memory 1101, so as to: in response to a placement operation of a first virtual object in a target space, move the first virtual object to a target placement surface in the target space, wherein the target placement surface corresponds to a target adsorption surface on the first virtual object; in the process of the first virtual object moving toward the target placement surface, adjust the first virtual object to a first target posture, wherein the first target posture is a posture in which the target adsorption surface is placed on the target placement surface; and in the first target posture, place the first virtual object on the target placement surface with the target adsorption surface.

[0103] Optionally, before moving the first virtual object to the target placement surface in the target space, the processor 1102 is further used to: determine at least one placement surface in the target space; select, from the at least one placement surface, a candidate placement surface corresponding to each of the at least one adsorption surfaces of the first virtual object; determine a target adsorption surface from the at least one adsorption surface according to the placement properties of the first virtual object and / or the distance between the at least one adsorption surface and the corresponding candidate placement surfaces; and determine the target placement surface according to the candidate placement surfaces corresponding to the target adsorption surface.

[0104] Optionally, when the processor 1102 determines the target adsorption surface from the at least one adsorption surface according to the placement properties of the first virtual object and / or the distance between the at least one adsorption surface and the corresponding candidate placement surfaces, the processor 1102 is specifically used to: determine whether there is an adsorption surface among the at least one adsorption surface that is adapted to the placement properties of the first virtual object; if so, use the adsorption surface that is adapted to the placement properties of the first virtual object as the target adsorption surface; if not, select the adsorption surface corresponding to the minimum distance according to the distance between the at least one adsorption surface and the respective candidate placement surfaces as the target adsorption surface.

[0105] Optionally, when the processor 1102 selects the candidate placement surface corresponding to each of the at least one adsorption surfaces from the at least one placement surface, it is specifically used to: determine, for any first adsorption surface among the at least one adsorption surface, a detection ray of the first adsorption surface to the at least one placement surface, wherein the direction of the detection ray of the first adsorption surface to any placement surface does not change with the posture change of the first virtual object; detect the at least one placement surface using the detection ray of the first adsorption surface to the at least one placement surface to obtain the distance between the first adsorption surface and the at least one placement surface; and determine, from the at least one placement surface, a candidate placement surface corresponding to the first adsorption surface according to the distance between the first adsorption surface and the at least one placement surface.

[0106] Optionally, when determining the detection ray of the first adsorption surface to the at least one placement surface, the processor 1102 is specifically used to: for any first placement surface among the at least one placement surface, determine, according to the posture of the first placement surface, a second target posture of the first virtual object placed on the first placement surface with the first adsorption surface; obtain a posture frame corresponding to the first virtual object under the second target posture, the posture frame including a final adsorption surface corresponding to the first adsorption surface; and use a vertical ray pointed from a point on the final adsorption surface to the first placement surface as a detection ray of the first adsorption surface to the first placement surface.

[0107] Optionally, the at least one placement surface includes: a virtual placement surface on a second virtual object displayed in the target space, a collision body being arranged on the outside of the second virtual object; optionally, when the processor 1102 detects the at least one placement surface using the detection ray of the first adsorption surface to the at least one placement surface and obtains the distance between the first adsorption surface and the at least one placement surface, it is specifically used to: use the detection ray of the first adsorption surface to the virtual placement surface to perform collision detection on the collision body of the virtual placement surface and obtain the distance between the first adsorption surface and the virtual placement surface.

[0108] Optionally, the at least one adsorption surface includes: at least one of a top surface, a bottom surface, and side surfaces in different directions of the first virtual object.

[0109] Optionally, when the processor 1102 adjusts the first virtual object to a first target posture during the process of the first virtual object moving toward the target placement surface, the processor 1102 is specifically used to: determine, according to the posture of the target placement surface, a posture of the first virtual object placed on the target placement surface with the target adsorption surface as the first target posture; during the process of the first virtual object moving toward the target placement surface, determine, according to the first target posture and the placement progress of the first virtual object, a real-time posture of the first virtual object at a plurality of consecutive moving moments; and adjust the first virtual object according to the real-time postures at the plurality of consecutive moments until the first virtual object is adjusted to the first target posture.

[0110] Optionally, when the processor 1102 determines the real-time posture of the first virtual object at a plurality of consecutive moving moments according to the first target posture and the placement progress of the first virtual object, the processor 1102 is specifically used to: determine the posture of the first virtual object before the placement operation as the initial posture; for any one of the plurality of consecutive moving moments, determine the placement progress of the first virtual object at the moving moment according to the first target posture and the distance between the first virtual object and the target placement surface at the moving moment; determine the posture change amount corresponding to the initial posture at the moving moment according to the placement progress; and update the initial posture according to the posture change amount to obtain the real-time posture of the first virtual object at the moving moment.

[0111] Optionally, the processor 1102 is further used to: determine the placement progress of the first virtual object according to the distance between the first virtual object and the target placement surface during the process of the first virtual object moving toward the target placement surface; and display the placement projection of the first virtual object in the target space according to the placement progress of the first virtual object.

[0112] Optionally, the processor 1102 is also used to: display a third virtual object in the target space, wherein a collision body is arranged outside the third virtual object; when a collision operation between the first virtual object and the third virtual object is detected, obtain a first position of the first virtual object when the collision occurs; after the collision operation, if it is detected that the first virtual object moves toward an internal space of the third virtual object, then when the first virtual object stops moving, obtain a second position of the first virtual object; determine a rebound path from the second position to a specific position before the collision operation based on a ray pointing from the second position to the first position; and display the movement state of the first virtual object in the target space based on the rebound path.

[0113] In this embodiment, in response to the placement operation of the first virtual object in the target space, the first virtual object is moved to the target placement surface in the target space. In the process of the first virtual object moving to the target placement surface, the first virtual object can be adjusted to a first target posture. After the first virtual object is adjusted to the first target posture, the target adsorption surface can be placed on the target placement surface under the first target posture. Among them, the first target posture is a posture in which the target adsorption surface is placed on the target placement surface. Based on this embodiment, before the first virtual object contacts the target adsorption surface, the posture of the first virtual object is adjusted to a posture in which the target adsorption surface is placed on the target placement surface, which can reduce the risk of spatial position conflict between the first virtual object and the target placement surface during the adsorption process of the target adsorption surface and the target placement surface, thereby reducing the "mold penetration" phenomenon in the mixed reality interaction process to a certain extent and enhancing the realism of the object placement operation in the target space. Fig.11 The electronic device can also be used to execute the following object processing method, wherein the processor 1102 is specifically used to: in response to a placement operation of a first virtual object in a target space, move the first virtual object toward a target placement surface in the target space, wherein the target placement surface is detected by a detection ray of a target adsorption surface on the first virtual object, and the direction of the detection ray is adapted to a target posture of the first virtual object being placed on the target placement surface with the target adsorption surface; when the first virtual object moves to an adsorption position corresponding to the target placement surface, rotate the first virtual object to a first target posture, wherein the first target posture is a posture in which the target adsorption surface is placed on the target placement surface.

[0114] Further, if Fig.11 As shown, the electronic device also includes: a communication component 1103, a power component 1104, a display component 1105, an audio component 1106, an image acquisition component 1107 and other components. Fig.11 Only some components are shown schematically, which does not mean that the electronic device only includes Fig.11 For example, in some embodiments, the electronic device may further include: an infrared sensor, a depth sensor, an inertial measurement unit, an ambient light sensor, a temperature sensor, a magnetic sensor, etc., which are not shown one by one.

[0115] Among them, the memory 1101 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0116] The communication component 1103 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as Wi-Fi (wireless network communication technology), 2G (such as Global System for Mobile Communications (GSM)), 3G (such as Wideband Code Division Multiple Access (WCDMA), 4G (such as Long Term Evolution (LTE)), 4G+ (such as upgraded Long Term Evolution (LTE-Advanced, LTE-A)) or 5G (5th Generation Mobile Communication Technology), or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can be implemented based on Near Field Communication (NFC) technology, Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0117] The power supply component 1104 is used to provide power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing and distributing power to the device where the power supply component is located.

[0118] The display assembly includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0119] The audio component may be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0120] The image acquisition component is used to collect scene information corresponding to the real space and collect the user's gesture interaction actions in the real space, etc. The image acquisition component can be implemented as one or more cameras.

[0121] In this embodiment, in response to the placement operation of the first virtual object in the target space, a target adsorption surface can be selected from the adsorption surface of the first virtual object, and a target placement surface corresponding to the target adsorption surface can be determined in the target space. In the process of the first virtual object moving toward the target placement surface, the first virtual object can be adjusted to a first target posture. After the first virtual object is adjusted to the first target posture, the target adsorption surface can be placed on the target placement surface under the first target posture. Among them, the first target posture is a posture in which the target adsorption surface is placed on the target placement surface. Based on this implementation, before the first virtual object contacts the target adsorption surface, the posture of the first virtual object is adjusted to a posture in which the target adsorption surface is placed on the target placement surface. During the adsorption process of the target adsorption surface and the target placement surface, the risk of spatial position conflict between the first virtual object and the target placement surface can be reduced, thereby reducing the "mold penetration" phenomenon in the mixed reality interaction process to a certain extent, and enhancing the realism of the object placement operation in the target space.

[0122] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be executed by an electronic device in the above method embodiment.

[0123] The embodiment of the present application also provides a computer program product, including: a computer program / instructions, which, when executed by a processor, can implement the steps in the method provided in the embodiment of the present application.

[0124] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM (Compact Disc Read-Only Memory, read-only optical disc read-only memory), optical storage, etc.) containing computer-usable program codes.

[0125] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0126] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0128] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0129] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0130] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, Parallel Random Access Machine (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0131] It should also be noted that the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, product or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, product or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, product or device including the element.

[0132] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for processing space objects, characterized in that: include: In response to a placement operation of a first virtual object in a target space, move the first virtual object toward a target placement surface in the target space, the target placement surface corresponding to a target adsorption surface on the first virtual object; During the process of the first virtual object moving toward the target placement surface, adjusting the first virtual object to a first target posture, wherein the first target posture is a posture in which the target adsorption surface is placed on the target placement surface; In the first target posture, the first virtual object is placed on the target placement surface with the target adsorption surface.

2. The method according to claim 1, characterized in that Before moving the first virtual object to the target placement surface in the target space, the method further includes: determining at least one placement surface in the target space; Selecting, from the at least one placement surface, candidate placement surfaces corresponding to the at least one adsorption surface of the first virtual object; Determine a target adsorption surface from the at least one adsorption surface according to the placement attribute of the first virtual object and / or the distance between the at least one adsorption surface and the corresponding candidate placement surface; The target placement surface is determined according to the candidate placement surface corresponding to the target adsorption surface.

3. The method according to claim 2, characterized in that Determining a target adsorption surface from the at least one adsorption surface according to the placement attribute of the first virtual object and / or the distance between the at least one adsorption surface and the corresponding candidate placement surface, comprises: Determining whether there is an adsorption surface among the at least one adsorption surface that is adapted to the placement attribute of the first virtual object; If so, taking the adsorption surface that matches the placement attribute of the first virtual object as the target adsorption surface; If it does not exist, then according to the distance between the at least one adsorption surface and the respective candidate placement surfaces, the adsorption surface corresponding to the minimum distance is selected as the target adsorption surface.

4. The method according to claim 2, characterized in that: Selecting, from the at least one placement surface, a candidate placement surface corresponding to each of the at least one adsorption surfaces, comprises: For any first adsorption surface among the at least one adsorption surface, determining a detection ray from the first adsorption surface to the at least one placement surface, wherein the direction of the detection ray from the first adsorption surface to any placement surface does not change with the posture change of the first virtual object; Using the detection ray of the first adsorption surface to the at least one placement surface, the at least one placement surface is detected to obtain the distance between the first adsorption surface and the at least one placement surface; According to the distance between the first adsorption surface and the at least one placement surface, a candidate placement surface corresponding to the first adsorption surface is determined from the at least one placement surface.

5. The method according to claim 4, characterized in that Determining the detection ray of the first adsorption surface to the at least one placement surface includes: For any first placement surface among the at least one placement surface, determining, according to the posture of the first placement surface, a second target posture of the first virtual object placed on the first placement surface with the first adsorption surface; Acquire a posture frame corresponding to the first virtual object in the second target posture, wherein the posture frame includes a final adsorption surface corresponding to the first adsorption surface; A vertical ray directed from a point on the final-state adsorption surface to the first placement surface is used as a detection ray from the first adsorption surface to the first placement surface.

6. The method according to claim 4, characterized in that The at least one placement surface comprises: a virtual placement surface on a second virtual object displayed in the target space, a collision body being arranged outside the second virtual object; Using the detection ray of the first adsorption surface to the at least one placement surface to detect the at least one placement surface, and obtaining the distance between the first adsorption surface and the at least one placement surface, comprises: The detection ray of the first adsorption surface to the virtual placement surface is used to perform collision detection on the collision body of the virtual placement surface to obtain the distance between the first adsorption surface and the virtual placement surface.

7. The method according to any one of claims 2 to 6, characterized in that: The at least one adsorption surface includes: at least one of a top surface, a bottom surface, and side surfaces in different directions of the first virtual object.

8. The method according to any one of claims 1 to 6, characterized in that: During the process of the first virtual object moving toward the target placement surface, adjusting the first virtual object to a first target posture includes: According to the posture of the target placement surface, determining a posture of the first virtual object placed on the target placement surface with the target adsorption surface as the first target posture; In the process of the first virtual object moving toward the target placement surface, determining the real-time posture of the first virtual object at a plurality of consecutive moving moments according to the first target posture and the placement progress of the first virtual object; The first virtual object is adjusted according to the real-time postures at the plurality of consecutive moments until the first virtual object is adjusted to the first target posture.

9. The method according to claim 8, characterized in that Determining the real-time posture of the first virtual object at a plurality of consecutive moving moments according to the first target posture and the placement progress of the first virtual object includes: determining a posture of the first virtual object before the placement operation as an initial posture; For any moving moment among the plurality of consecutive moving moments, determining the placement progress of the first virtual object at the moving moment according to the first target posture and the distance between the first virtual object and the target placement surface at the moving moment; According to the placement progress, a posture change amount corresponding to the initial posture at the moving moment is determined; according to the posture change amount, the initial posture is updated to obtain a real-time posture of the first virtual object at the moving moment.

10. The method according to any one of claims 1 to 6, characterized in that: Also includes: During the process of the first virtual object moving toward the target placement surface, determining the placement progress of the first virtual object according to the distance between the first virtual object and the target placement surface; According to the placement progress of the first virtual object, a placement projection of the first virtual object is displayed in the target space.

11. The method according to any one of claims 1 to 6, characterized in that: Also includes: Displaying a third virtual object in the target space, wherein a collision body is arranged outside the third virtual object; When a collision operation between the first virtual object and the third virtual object is detected, obtaining a first position of the first virtual object when the collision occurs; After the collision operation, if it is detected that the first virtual object moves toward the inner space of the third virtual object, when the first virtual object stops moving, obtaining a second position of the first virtual object; determining a rebound path from the second position to a specific position before the collision operation according to a ray pointing from the second position to the first position; The moving state of the first virtual object is displayed in the target space according to the rebound path.

12. A method for processing space objects, characterized in that: include: In response to a placement operation of a first virtual object in a target space, move the first virtual object toward a target placement surface in the target space, the target placement surface being detected by a detection ray of a target adsorption surface on the first virtual object, the direction of the detection ray being adapted to a target posture of the first virtual object being placed on the target placement surface with the target adsorption surface; When the first virtual object moves to the adsorption position corresponding to the target placement surface, the first virtual object is rotated to a first target posture, where the first target posture is a posture in which the target adsorption surface is placed on the target placement surface.

13. An electronic device, characterized in that: include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute the one or more computer instructions to: perform the steps in the method according to any one of claims 1-12.

14. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps in the method described in any one of claims 1 to 12 can be implemented.

15. A computer program product, characterized in that include: A computer program / instruction, which, when executed by a processor, can implement the steps of the method according to any one of claims 1 to 12.

Images