Method, device, equipment and storage medium for augmented reality interaction
By presenting the indicator information of the target object in the AR scene and updating the rendering of components based on the user equipment pose, the problem of insufficient efficiency and effect of existing AR interaction technology is solved, and better user experience and interaction effects are achieved.
Patent Information
- Application Number
- CN202211141379.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-20
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-09-20
AI Technical Summary
The existing augmented reality (AR) interaction technology has insufficient efficiency and effectiveness in the interaction mode in the scenario, affecting the user experience.
Users are allowed to interact and manipulate by presenting the indication information of the target object in the AR scene of the user device and updating the rendering of the component based on changes in the user device's posture until the component visually matches the indication information and rendering the target object as a whole.
It improves the user experience of AR interaction, provides an interesting and challenging interactive process, and enhances the interactive experience between users and virtual objects.
Smart Images

Figure CN115454250B_ABST
Abstract
Description
Technical Field
[0001] Example embodiments of the present disclosure relate generally to augmented reality (AR), and more particularly, to methods, apparatuses, devices, and computer-readable storage media for interacting in an AR scene. Background Art
[0002] Augmented Reality (AR) technology is a technology that integrates virtual information with the real world. AR devices can superimpose virtual objects with images in the real world and present them in AR scenes. In this way, the images that appear in the user's field of view include both real-world images and virtual objects. In other words, AR scenes allow users to see virtual objects and the real world at the same time, giving users an immersive experience. AR has a wide range of applications in many fields such as games, simulations, and simulations. Various interactive technologies allow users to interact with virtual objects or objects in AR scenes. The methods, efficiency, and effects of these interactive technologies directly affect the user's experience of using AR devices. Summary of the invention
[0003] In a first aspect of the present disclosure, a method for augmented reality (AR) interaction is provided. The method comprises: presenting indication information of a target object and one or more components of the target object in an AR scene presented by a user device; updating the rendering of the one or more components in the AR scene based on a change in the posture of the user device; and presenting the target object as a whole in the AR scene in response to the updated one or more components visually matching the indication information.
[0004] In a second aspect of the present disclosure, a device for augmented reality (AR) interaction is provided. The device includes: a presentation control module configured to present indication information of a target object and one or more components of the target object in an AR scene presented by a user device; a rendering update module configured to update the rendering of one or more components in the AR scene based on a change in the posture of the user device; and an object interaction module configured to present the target object as a whole in the AR scene in response to the updated one or more components visually matching the indication information.
[0005] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the electronic device executes the method according to the first aspect of the present disclosure.
[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to perform the method according to the first aspect of the present disclosure.
[0007] It should be understood that the contents described in the summary of the present invention are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] In the following, in conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages and aspects of various implementations of the present disclosure will become more apparent. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:
[0009] Figure 1 A schematic diagram showing an example environment in which various embodiments of the present disclosure can be implemented;
[0010] Figure 2 A schematic diagram illustrating the principle of "near is big and far is small" is shown;
[0011] Figure 3 A flowchart of a method for AR interaction according to some embodiments of the present disclosure is shown;
[0012] Figure 4A and Figure 4B Schematic diagrams respectively showing several examples of presenting indication information of a target object in an AR scene according to some embodiments of the present disclosure;
[0013] Figure 4C and Figure 4D Schematic diagrams respectively showing rendering updates of one or more components as the posture of a user device changes according to some embodiments of the present disclosure;
[0014] Figure 4E A schematic diagram showing when one or more components visually match the indication information of a target object according to some embodiments of the present disclosure;
[0015] Figure 4F A schematic diagram showing how to present and manipulate a target object as a whole according to some embodiments of the present disclosure is shown;
[0016] Figure 5A , Figure 5B and Figure 5C A schematic diagram showing another example of operating components of a target object to match a target object contour according to some embodiments of the present disclosure;
[0017] Figure 6A block diagram showing an apparatus for augmented reality according to some embodiments of the present disclosure; and
[0018] Figure 7 A block diagram of a device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION
[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0020] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.
[0021] The term "in response to" means that the corresponding event occurs or the condition is satisfied. It will be understood that the timing of the execution of the subsequent action executed in response to the event or condition is not necessarily strongly related to the time when the event occurs or the condition is satisfied. In some cases, the subsequent action may be executed immediately when the event occurs or the condition is satisfied; in other cases, the subsequent action may be executed some time after the event occurs or the condition is satisfied.
[0022] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.
[0023] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0024] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can independently choose whether to provide personal information to software or hardware such as electronic devices, applications, servers or storage media that execute operations of the technical solution of the present disclosure based on the prompt message.
[0025] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information is sent to the user in a manner such as a pop-up window, in which the prompt information can be presented in text form. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0026] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.
[0027] The embodiments of the present disclosure are described below in conjunction with the accompanying drawings. It will be understood from the following description that, according to the embodiments of the present disclosure, a scheme for users to interact with virtual objects in an AR scene is provided. In the AR scene, indication information about the target virtual object and one or more components of the virtual object are presented to the user at the same time. Initially, these components are visually scattered, that is, they are not assembled into the target virtual object. The user can change the posture (e.g., orientation, tilt, distance, etc.) of his user device to change the field of view (FoV) of the AR virtual camera used for the AR scene. Accordingly, the rendering effects of these components in the AR scene will also be refreshed. When the user device reaches a predetermined posture, visually, one or more components rendered in the AR scene will match the target virtual object. At this point, the "assembly" operation can be triggered, that is, these scattered components are rendered as a target virtual object as a whole. Then, the user can interactively manipulate the target virtual object as a whole. In this way, the user can obtain an interesting interaction process and enhance the user experience of the AR environment.
[0028] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In the example environment 100, an AR scene 150 is presented to a user 130 at or by a user device 110. The AR scene 150 may be presented on a screen of the user device 110. The AR scene 150 may include a real-world screen 154 and virtual objects 1531 and 1532 superimposed on the screen 154.
[0029] In screen 154, objects 1541 and 1542 are representations of real objects in the real world in AR scene 150, such as images or other forms of representations of real objects. For ease of discussion only, objects 1541 and 1542 are also referred to herein as 3D objects. When the posture of user device 110 changes, the field of view of the AR camera used for the AR scene also changes. Accordingly, screen 154 will change as the posture of user device 110 changes. Accordingly, the rendering of 3D objects presented in screen 154 will also be updated, causing the visual effects of these 3D objects to change accordingly. Similarly, the rendering of virtual objects 1531 and 1532 will also be updated, causing the visual effects of these virtual objects to change accordingly.
[0030] In some embodiments, the user device 110 may include a positioning component. The positioning component is used to obtain the posture of the user device 110, and determine the position of the objects and the virtual camera in the AR scene in real time based on the posture. In this way, the rendering can be updated as the posture of the user device 110 changes.
[0031] It should be understood that the AR scene 150 is merely exemplary and is not intended to limit the scope of the present disclosure. The AR scene 150 may include more or fewer virtual objects superimposed on the screen 154, or may include other elements, such as user interface (UI) elements.
[0032] The user device 110 may be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a gaming device, a wearable device, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the user device 110 may also support any type of interface for the user (such as a "wearable" circuit, etc.).
[0033] The user device 110 may have an AR engine 120 installed thereon. The AR engine 120 is used to drive the presentation of the AR scene 150. In some embodiments, the AR engine 120 may be an AR game engine; accordingly, the AR scene 150 may be an AR game scene. In some embodiments, the AR engine 120 may be part of a content sharing application (or "social application"). The social application can provide services related to multimedia content consumption to the user 130, such as allowing users to publish, watch, comment, forward, create multimedia works, and so on. Accordingly, the AR scene 150 may be an AR content creation scene. For example, in some embodiments, the AR scene 150 may be part of the special effects provided by the social application. The positioning component described above may be part of the AR engine 120.
[0034] It should be understood that the structure and functionality of environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure. User device 110 may include any suitable structure and functionality to enable interaction with an AR scene.
[0035] It can be understood that according to the perspective principle, when the field of view of the user or virtual camera is different, the visual effect of the 3D object presented in the AR scene will also change. Take the human eye as an example. The lens in the eye is equivalent to a convex lens, and the retina is equivalent to a surface. The human eye sees an object through the image of the object on the retina. The angle of the object from the center of the pupil is equal to the viewing angle. Therefore, the size of the viewing angle determines the size of the image of the object on the retina. When an object is far from the eye, the viewing angle is relatively small, so the size appears to be smaller; when the same object is close to the eye, the viewing angle becomes larger, and the visual effect appears to be larger. The size of the object has not changed, but because of the different viewing angles, the size of the visual effect has changed. This is the visual principle of "near big and far small". The camera also follows this principle in shooting and imaging.
[0036] Using this principle, we can achieve some visual effects that would not happen in the real world. For example, Figure 2 The scene 200 captured by the camera is shown. In the scene 200, because the hand is close to the viewpoint, the image 202 of the hand looks relatively large; and the sun is far from the viewpoint, so the image 204 of the sun looks very small. This enables the effect of "pinching the sun with hands" to be achieved.
[0037] Similarly, due to different viewpoints and / or perspectives, the visual effects of the same object rendered in an AR scene may vary. According to the embodiments of the present disclosure, these visual and imaging principles are fully utilized in the process of supporting users to interact with 3D objects in an AR scene, and a set of interaction mechanisms that can enable users to obtain a good experience is provided.
[0038] Figure 3 A flowchart of a method 200 for AR interaction according to some embodiments of the present disclosure is shown. In some embodiments, the method 200 may be, for example, Figure 1 The AR engine 120 or other suitable modules / devices may be implemented at the user equipment 110 as shown.
[0039] In block 302 , in an AR scene presented by the user device 110 , indication information of a target object and one or more components of the target object are presented.
[0040] Figure 4A A schematic diagram showing the presentation of indication information of a target object and its components in an AR scene 400 presented by a user device 110 is shown. As shown in the figure, the AR scene 400 includes a real world portion 402 captured by a camera of the user device 110 and a virtual object. In this example, the real world portion 402 is a bookshelf and is used as a background portion of the scene.
[0041] A virtual object includes one or more components of a target object. Figure 4A In the example shown, the target object (or "target virtual object") is a snowman. Multiple components 406, 408, and 410 of the target object are presented in the AR scene 400. More specifically, component 406 is the snowman's hat, component 408 is the snowman's head, and component 410 is the snowman's body. Note that in some embodiments, only one component of the target object may be presented, such as the main or largest component.
[0042] The AR scene 400 also presents information indicating the target object, the snowman. Figure 4A In the example shown, the indication information is the outline 404 of the target object "snowman". The outline 404 is presented as a contour line. In some embodiments, the outline 404 can be presented in the AR scene 400 at a predetermined size and / or position. That is, when the user changes the posture of the user device 110, thereby causing the field of view of the AR scene 400 to change, the position and size of the outline 404 on the user interface of the user device 110 can remain unchanged. Alternatively, in other embodiments, the outline 404 can also change position and / or size as the scene changes. It is difficult for the user to distinguish the overall appearance of the target object to be assembled just by seeing the disassembled components. By presenting the outline of the target object to the user, the target of the splicing of the fragments can be clearly understood. In this way, the user experience can be further improved.
[0043] Note that the contour is only one example of the indication information of the target object. Other suitable forms of indication information are also possible. Figure 4B In the example shown, other rendering content is similar to Figure 4A , but the target object indication information is changed from the outline to icon 412. The appearance of icon 412 is a snowman, thereby indicating to the user that the current target object is a snowman. In addition to or in place of the icon, text, animation, etc. may also be displayed as the target object indication information. The content of the present disclosure is not limited in this respect.
[0044] return Figure 3 , in box 304, based on the change in the posture of the user device 110, the rendering of one or more components of the target object in the AR scene is updated. Specifically, during operation, the user can change the various posture parameters of his user device 110, such as the orientation, angle, and position. The change in the posture of the user device 110 will cause the field of view of the AR camera used for the AR scene to change. Accordingly, in the rendering result presented to the user via the user interface, the visual effect of the 3D objects in the scene will also change. For example, the positioning component can obtain the posture of the user device 110 in real time, thereby changing the position of the AR camera and the coordinates of one or more components. The rendering of one or more components also changes accordingly.
[0045] Figure 4C and Figure 4D Schematic diagrams showing rendering updates of one or more components as the posture of the user device 110 changes according to some embodiments of the present disclosure are shown respectively. Figure 4C In the schematic diagram shown, it can be seen that as the user changes the orientation of the user device 110 in the direction indicated by the arrow 414, the rendering of the 3D object in the AR scene 400 is updated accordingly. As the field of view of the camera of the user device 110 changes, the scene contained in the background part (i.e., the image of the real world scene) is changed compared to the background part. Figure 4A-4B There has been a change.
[0046] At the same time, the rendering of components 406-408 of the target object is also updated as the AR camera's field of view changes. Specifically, according to the principle that objects are larger when they are closer and smaller when they are farther away, component 406 becomes visually smaller, while component 410 becomes visually larger. Moreover, because the AR camera's perspective changes, the visible parts of each component may also change accordingly. That is, some previously invisible parts may become visible; some previously visible parts may no longer be visible.
[0047] Furthermore, according to the principles of vision and imaging, when the field of view of the AR camera is different, the visual relative position relationship of multiple objects whose actual relative relationship remains unchanged may change. According to this principle, as the posture of the user device 110 changes, the visual position relationship between the various components may also change due to the change of the perspective relationship in the AR scene. Specifically, in the example shown, Figure 4A and 4B Compared to the state shown in FIG. 4 , the distance between components 406 - 410 appears to have changed. Note that this is only a visual change caused by the rendering update, and their actual relative position relationship in the scene has not changed.
[0048] Figure 4D 4 shows an update of the rendering of the target object components 406-410 caused by the user further changing the posture of the user device 110 along the direction 416. Figure 4D In the example shown, the user pans the user device 110 to the left, causing the updated rendered components 406-410 to appear larger in the AR scene 400. Figure 4C To the right, thereby getting closer to the indication of the target object (specifically, the outline 404 in this example).
[0049] In summary, in operation, the field of view of the AR camera can be determined according to the posture of the user device 110, and the components 406-410 of the target object can be rendered in the AR scene 400 according to the field of view. Figure 4C and Figure 4D In the example shown, the outline 404 does not change its appearance as the posture of the user device 110 changes. That is, the outline 404 remains at a fixed position and size in the user interface. Alternatively, in other embodiments, the outline 404 may also appropriately change its position and size as the posture of the user device 110 changes, but the degree of change may be different from that of the components 406-410.
[0050] It will be appreciated that as the user adjusts the presentation of the one or more components 406-410 of the target object by changing the device posture, in a certain state, the updated one or more components 406-410 visually match the indication information of the target object. Figure 4E In the illustrated state, the overall outline formed by the components 406-410 visually matches the outline 404 of the target object, that is, within a given tolerance range, the two are consistent with each other in shape and size.
[0051] This visual match can be detected by various appropriate algorithms. For example, in some embodiments, the AR engine 120 can pre-calculate and store a target position associated with the target object. When the position of the AR camera associated with the AR scene is close to or aligned with the target position, it is considered that the multiple components of the target object visually fall exactly within the range defined by the outline. In operation, as the posture of the user device 110 changes, the AR engine 120 determines the current position of the AR camera accordingly, and compares the current position with the target position.
[0052] The camera position can be characterized in various appropriate ways, such as by the camera's position, height, distance, orientation, etc. If the difference between the current camera position of the AR camera and the predetermined target camera position is lower than a predetermined threshold, it can be determined that one or more components visually match the outline. For example, if the position and rotation angle orientation of the current camera position of the AR camera and the target camera position are within a certain error range, the two camera positions can be considered close, so that one or more components can be determined to visually match the outline.
[0053] It will be understood that in other embodiments where the indication information is not an outline, the above-mentioned camera position matching method is also applicable. That is, when the user operates the user device 110 to make the AR camera reach a predetermined target camera position, it is considered that the overall effect of these components visually formed matches the displayed icon or text.
[0054] However, it should be understood that such matching of the predetermined target camera position is not necessary, but is merely exemplary. For example, in some alternative embodiments, after the rendering of the components is updated, the maximum external bounding box of these components can be calculated, and the bounding box can be matched with the contour as the target object indication information to detect visual matching. Other matching methods are also possible, and the present disclosure is not limited in this respect.
[0055] Figure 4E A schematic diagram is shown when components 406-410 visually match the indication information of the target object according to some embodiments of the present disclosure. In this example, it can be seen that components 406-410 as a whole visually fall within contour 404.
[0056] return Figure 3 In response to the above matching, at block 306, the target object as a whole is presented in the AR scene. Figure 4F , a complete target object 418, i.e., a snowman, is drawn in the AR scene 400 as a whole. At this time, the components of the snowman are no longer separate and independent individuals. Instead, they are presented and drawn as one object or virtual object in the AR scene 400.
[0057] For example, when the user changes the posture of the user device 110 again, the drawing of the assembled snowman will be updated accordingly, but will no longer be presented as Figure 4C and 4D That is, after the components of the target object visually match the indication information of the target object, the AR engine 120 no longer updates the rendering of the components 406-410 as the posture of the user device 110 changes; instead, it only renders and updates the target object 418 as a whole.
[0058] In some embodiments, if a positioning component is used, the positioning component can be disabled or turned off in response to the above match. This means that changes in the user's posture of the user device 110 (e.g., user shaking) will not affect the position of the AR camera. Instead, the position of the AR camera is aligned with the target position through the position hosting calibration. As a result, the target object can be rendered as a whole.
[0059] In this embodiment, when the position of the AR camera is close to the target position, the target object can be presented as a whole. In this way, it can be avoided that the user needs to spend a long time to reach the object assembly target due to slight shaking of the user device (for example, the user's hand shaking). This further improves the user experience.
[0060] Alternatively or additionally, in some embodiments, after one or more components are "assembled" into a target object at block 306, the user can manipulate the target object 418 as a whole, such as rotating, scaling, dragging, etc. For example, after the above-mentioned managed calibration is completed, the AR component can be enabled to return the user's control authority. Later, the user can interact with the target object as a whole.
[0061] In some embodiments, once the updated component visually matches the indication information of the target object, the AR engine 120 may also stop drawing the indication information of the target object, such as its outline information, in the AR scene 400. Figure 4F , outline 404 is no longer displayed.
[0062] Some embodiments of the present disclosure have been described above with reference to several example scenarios. According to an embodiment of the present disclosure, a user can visually assemble one or more independent components in an AR scene into a complete target object by adjusting the posture of his or her user device. The entire interactive process is full of challenges and fun, and can be widely used in multiple fields such as AR special effects, cognitive behavioral training and assistance, simulation, and games for social applications to improve the user's interactive experience.
[0063] Figure 5A , Figure 5B and Figure 5C A schematic diagram of another example of operating components of a target object to match the target object contour according to some embodiments of the present disclosure is shown. In the example shown here, the AR scene 500 also has a background portion 502, which is an image of a real-world scene captured by a camera of the user device 110, in this example, the background portion.
[0064] In addition to the background portion 502, as Figure 5A As shown, first, the outline 504 of the target object "hamburger" and multiple components 506, 508, 510, 512, 514 of the target object are presented to the user in the AR scene 500. More specifically, in this example, these components are the bread slices, meat patties, cheese slices, and vegetables that make up the hamburger.
[0065] like Figure 5B As shown, the user adjusts the posture of the user device 110 so that the components 506, 508, 510, 512, and 514 visually fall exactly within the outline 504, and the difference between the boundaries of the two is within a predetermined tolerance range. As described above, in some embodiments, this can be detected, for example, by detecting whether the position of the AR camera reaches a predetermined target position.
[0066] like Figure 5C As shown, once it is determined that the components 506, 508, 510, 512, 514 visually match the outline 504, the positioning component is disabled to transfer the user's control. That is, the target object "hamburger" 516 is then updated as a whole for rendering and / or responding to user operations, and the individual components 506, 508, 510, 512, 514 can no longer be independently controlled. At the same time, the outline 504 stops being displayed.
[0067] Figure 6 A block diagram of an apparatus 600 for AR interaction is shown. As shown, the apparatus 600 includes a presentation control module 610, which is configured to present indication information of a target object and one or more components of the target object in an AR scene presented by a user device. The apparatus 600 also includes a rendering update module 620, which is configured to update the rendering of the one or more components in the AR scene based on a change in the posture of the user device. The apparatus 600 further includes an object interaction module 630, which is configured to present the target object as a whole in the AR scene in response to the updated one or more components visually matching the indication information.
[0068] In some embodiments, the presentation control module 610 is configured to present the outline of the target object in a predetermined size and / or position in the AR scene.
[0069] In some embodiments, the presentation control module 610 is further configured to stop presentation of the outline in the AR scene in response to the updated one or more components visually matching the indication information.
[0070] In some embodiments, the object interaction module 630 is further configured to: in response to receiving an input for the target object after presenting the target object as a whole, manipulate the target object as a whole according to the input.
[0071] In some embodiments, the device 600 also includes a visual matching module (not shown), which is configured to determine whether the one or more components visually match the indication information as follows: determine the current position of the AR camera used for the AR scene based on the posture of the user device; compare the current position with a predetermined target position associated with the target object; and in response to the difference between the current position and the target position being lower than a predetermined threshold, determine that the one or more components visually match the indication information.
[0072] In some embodiments, the rendering update module 620 is configured to: determine the field of view of the AR camera for the AR scene based on the posture of the user device; and render the one or more components in the AR scene based on the field of view of the AR camera.
[0073] In some embodiments, the rendering update module 620 is further configured to: in response to the updated one or more components visually matching the indication information, stop updating the rendering of the one or more components as the posture of the user device changes.
[0074] The units included in the device 600 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units in the device 600 can be implemented at least in part by one or more hardware logic components. As an example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0075] Figure 71 shows a block diagram of a computing device 700 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 7 The computing device 700 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 7 The computing device 700 shown may be used to implement Figure 1 The user equipment 110.
[0076] like Figure 7 As shown, computing device 700 is in the form of a general electronic device. Components of computing device 700 may include, but are not limited to, one or more processors or processing units 710, memory 720, storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. Processing unit 710 may be an actual or virtual processor and is capable of performing various processes according to a program stored in memory 720. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capabilities of computing device 700.
[0077] The computing device 700 typically includes a plurality of computer storage media. Such media may be any available media accessible to the computing device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 720 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 730 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the computing device 700.
[0078] The computing device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 7 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 720 may include a computer program product 725 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.
[0079] The communication unit 740 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the computing device 700 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Therefore, the computing device 700 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.
[0080] Input device 750 may be one or more input devices, such as a mouse, keyboard, tracking ball, etc. Output device 760 may be one or more output devices, such as a display, a speaker, a printer, etc. Computing device 700 may also communicate with one or more external devices (not shown) through communication unit 740 as needed, such as storage devices, display devices, etc., communicate with one or more devices that allow a user to interact with computing device 700, or communicate with any device that allows computing device 700 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).
[0081] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which one or more computer instructions are stored, wherein the one or more computer instructions are executed by a processor to implement the method described above.
[0082] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0083] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0084] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0085] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.
[0086] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the marketplace, or to enable other persons of ordinary skill in the art to understand the implementations disclosed herein.
Claims
1. A method for augmented reality (AR) interaction, comprising: In the AR scene presented by the user device, presenting indication information of the target object and one or more components of the target object; Based on the change in the posture of the user device, updating the rendering of the one or more components in the AR scene; as well as In response to the updated one or more components visually matching the indication information, presenting the target object visually as a whole in the AR scene. 2 . The method according to claim 1 , wherein presenting the indication information comprises presenting an outline of the target object in the AR scene with a predetermined size and / or position.
3. The method according to claim 2, further comprising: In response to the updated one or more components visually matching the indication information, the presentation of the outline in the AR scene is stopped.
4. The method according to any one of claims 1 and 2, further comprising: In response to receiving an input for the target object after presenting the target object as a whole, the target object as a whole is manipulated according to the input.
5. The method according to any one of claims 1 and 2, further comprising determining whether the one or more components visually match the indication information as follows: Determining a current position of an AR camera for the AR scene according to the posture of the user equipment; comparing the current position with a predetermined target position associated with the target object; and In response to a difference between the current position and the target position being lower than a predetermined threshold, it is determined that the one or more components visually match the indication information.
6. The method according to any one of claims 1 and 2, wherein updating the rendering of the one or more components in the AR scene comprises: Determining a field of view of an AR camera for the AR scene according to a posture of the user device; as well as The one or more components are rendered in the AR scene according to the field of view of the AR camera.
7. The method according to any one of claims 1 and 2, further comprising: In response to the updated one or more components visually matching the indication information, stopping updating the rendering of the one or more components as the posture of the user device changes.
8. A device for augmented reality (AR) interaction, comprising: A presentation control module, configured to present indication information of a target object and one or more components of the target object in an AR scene presented by a user device; a rendering update module, configured to update the rendering of the one or more components in the AR scene based on a change in a posture of the user device; as well as The object interaction module is configured to present the target object as a visual whole in the AR scene in response to the updated one or more components visually matching the indication information. 9 . The device according to claim 8 , wherein the presentation control module is configured to present the outline of the target object in the AR scene at a predetermined size and / or position. 10 . The apparatus according to claim 9 , wherein the presentation control module is further configured to: in response to the updated one or more components visually matching the indication information, stop presentation of the outline in the AR scene.
11. The apparatus according to any one of claims 8 and 9, wherein the object interaction module is further configured to: in response to receiving an input for the target object after presenting the target object as a whole, manipulate the target object as a whole according to the input.
12. The apparatus according to any one of claims 8 and 9, further comprising a visual matching module configured to determine whether the one or more components visually match the indication information as follows: Determining a current position of an AR camera for the AR scene according to the posture of the user equipment; comparing the current position with a predetermined target position associated with the target object; and In response to a difference between the current position and the target position being lower than a predetermined threshold, it is determined that the one or more components visually match the indication information.
13. The apparatus according to any one of claims 8 and 9, wherein the rendering update module is configured to: Determining a field of view of an AR camera for the AR scene based on the posture of the user equipment; and The one or more components are rendered in the AR scene according to the field of view of the AR camera.
14. An apparatus according to any one of claims 8 and 9, wherein the rendering update module is further configured to: in response to the one or more components after update visually matching the indication information, stop updating the rendering of the one or more components as the posture of the user device changes.
15. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the device to perform the method according to any one of claims 1 to 7 when executed by the at least one processing unit.
16. A computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
A method and arrangement for handling haptic feedback
CN110100222A
Image data processing method and device
CN110275968A
Augmented reality data presentation method and device, display equipment and storage medium
CN111862340A
An interactive method and system for immersive content
CN112286355A
Virtual object detection method and device, and readable storage medium
CN112915539A
