An augmented reality display method, apparatus, and computer-readable storage medium
By displaying prompts and virtual objects in augmented reality and using sensor information for interactive control, the problem of users being unable to experience AR after leaving the AR range is solved, achieving efficient AR re-entry and improving the user experience.
Patent Information
- Application Number
- CN202211200826.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-29
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-09-29
AI Technical Summary
In augmented reality applications, users cannot experience AR when they leave the AR range. Existing text prompts are inefficient and result in a poor user experience.
The system displays virtual objects, virtual viewpoints, and augmented reality range through a prompt interface, and responds to sensor information for interactive control. When the interactive state of the virtual object matches the augmented reality range, the augmented reality display is activated, guiding the user's operation in a visual way.
It improves the efficiency of augmented reality displays, allowing users to intuitively understand how to re-enter the AR area, thus enhancing the user experience.
Smart Images

Figure CN117008713B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of human-computer interaction, specifically to an augmented reality display method, device, and computer-readable storage medium. Background Technology
[0002] Augmented Reality (AR) technology is a technology that cleverly integrates virtual information with the real world. It widely uses various technologies such as multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing to simulate and apply computer-generated text, images, 3D models, music, videos, and other virtual information to the real world. The two types of information complement each other, thereby achieving "enhancement" of the real world.
[0003] In related technologies, users can experience AR within the AR range constructed in AR applications, which has good fun and realism. However, in the process of researching and practicing related technologies, the inventors of this application found that when users leave the AR range, the AR experience cannot be performed. Only simple text prompts are given to guide users back to the AR range, which is very inefficient and results in a very poor user experience. Summary of the Invention
[0004] This application provides an augmented reality display method, apparatus, and computer-readable storage medium, which can improve the efficiency of augmented reality display and enhance the user experience based on visual prompts.
[0005] To address the aforementioned technical problems, this application provides the following technical solutions:
[0006] An augmented reality display method includes:
[0007] A prompt interface is displayed, on which a virtual object, the virtual viewpoint of the virtual object, and the augmented reality range are displayed;
[0008] Display interactive prompts on the prompt interface;
[0009] In response to the collected sensor information, the movement of the virtual object in the prompt interface and the orientation of the virtual viewpoint are interactively controlled;
[0010] When the interaction state of the virtual object is detected to match the augmented reality range, the augmented reality display is enabled.
[0011] An augmented reality display device, comprising:
[0012] The first display unit is used to display a prompt interface, and to display a virtual object, the virtual viewpoint of the virtual object, and the augmented reality range on the prompt interface;
[0013] The second display unit is used to display interactive prompt information on the prompt interface;
[0014] A control unit is used to interactively control the movement of the virtual object and the orientation of the virtual viewpoint in the prompt interface in response to the collected sensor information.
[0015] An activation unit is used to activate augmented reality display when the interaction state of the virtual object is detected to match the augmented reality range.
[0016] In some embodiments, the sensor information includes at least displacement data and steering data, and the control unit is configured to:
[0017] In response to displacement data in the sensor information, interactive control is performed on the movement of the virtual object;
[0018] In response to the steering data in the sensor information, the orientation of the virtual viewpoint is interactively controlled.
[0019] In some embodiments, the apparatus further includes an identification display unit for:
[0020] The first identifier is displayed on the prompt interface;
[0021] The first identifier is used to indicate that the virtual object is far from the augmented reality range.
[0022] In some embodiments, the apparatus further includes a determining unit for:
[0023] Displacement data is collected using a gyroscope;
[0024] Acquire the first orientation information of the camera, and generate turning data based on the first orientation information, wherein the first orientation information is used to indicate the orientation of the terminal;
[0025] Sensor information is determined based on the displacement and steering data.
[0026] In some embodiments, the display unit is further configured to:
[0027] Acquire first positioning information, second orientation information, and the location range of the augmented reality display; wherein the first positioning information is used to indicate the location of the terminal, and the second orientation information is used to indicate the orientation of the terminal;
[0028] Based on the location range, a corresponding augmented reality range is generated on the prompt interface;
[0029] The virtual position of the virtual object on the prompt interface is determined based on the first positioning information;
[0030] The virtual object is displayed at the virtual location, and a virtual view of the virtual object is generated based on the second orientation information.
[0031] In some embodiments, the apparatus further includes a feedback prompting unit for:
[0032] Generate vibration commands and generate vibration feedback based on the vibration commands;
[0033] The second display unit is used in at least one of the following steps:
[0034] A text box is displayed on the prompt interface, and interactive prompt information is displayed in the text box;
[0035] Interactive prompts are played via voice, and corresponding images are displayed on the prompt interface.
[0036] In some embodiments, the apparatus further includes a first pause unit, configured to:
[0037] Obtain the current second location information and the location range displayed by augmented reality, wherein the second location information is used to indicate the location of the terminal;
[0038] When it is detected that the location indicated by the second location information is not within the specified location range, the augmented reality display is paused and a prompt interface is displayed.
[0039] The opening unit is used for;
[0040] When the virtual position of the virtual object is detected to have shifted into the augmented reality range, the augmented reality display is activated.
[0041] In some embodiments, the apparatus further includes a first prompting unit, configured to:
[0042] Based on the virtual position of the virtual object, the orientation of the virtual viewpoint, and the relationship with the augmented reality range, a corresponding navigation trajectory and navigation prompts are generated;
[0043] The navigation trajectory is displayed on the virtual object, and the navigation prompts are read aloud via voice.
[0044] In some embodiments, the device further includes a vibration generating unit for:
[0045] Vibration data is generated, and vibration feedback of corresponding intensity is generated based on the vibration data;
[0046] The magnitude of the vibration data is proportional to the distance between the location indicated by the second positioning information and the location range.
[0047] In some embodiments, the apparatus further includes a second pause unit for:
[0048] Obtain the current third orientation information and the location range of the augmented reality display, wherein the third orientation information is used to indicate the orientation of the terminal;
[0049] When it is detected that the orientation indicated by the third orientation information does not point to the location range, the augmented reality display is paused and a prompt interface is displayed.
[0050] The opening unit is also used for;
[0051] When the virtual viewpoint of the virtual object is detected to be facing the augmented reality range, the augmented reality display is activated.
[0052] In some embodiments, the apparatus further includes a second prompting unit for:
[0053] Based on the relationship between the orientation of the virtual viewpoint and the augmented reality range, generate corresponding turning arrows and turning prompts;
[0054] The turning arrow is displayed on the virtual object, and the turning prompt is announced via voice.
[0055] A computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to perform the steps in the augmented reality display method described above.
[0056] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the augmented reality display method provided above.
[0057] A computer program product or computer program includes computer instructions stored in a storage medium. A processor of a computer device reads the computer instructions from the storage medium and executes the computer instructions, causing the computer device to perform the steps in the augmented reality display method provided above.
[0058] This application embodiment displays a prompt interface showing a virtual object, its virtual viewpoint, and its augmented reality range. Interactive prompts are displayed on the prompt interface. In response to collected sensor information, the movement of the virtual object and the orientation of the virtual viewpoint are interactively controlled. When the interactive state of the virtual object matches the augmented reality range, the augmented reality display is activated. In this way, the virtual object, virtual viewpoint, and augmented reality range are displayed visually on the prompt interface, allowing the user to move and turn based on the prompt interface, and simultaneously interactively control the movement and orientation of the virtual object. This achieves visual guidance. When the interactive state of the virtual object matches the augmented reality range, the augmented reality display is quickly activated. Compared to solutions that only provide simple text prompts to guide the user back to the AR range, users can intuitively understand how to activate the augmented reality display and operate efficiently based on visual guidance, greatly improving the efficiency of the augmented reality display. Attached Figure Description
[0059] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0060] Figure 1 This is a scene diagram of the augmented reality display system provided in the embodiments of this application;
[0061] Figure 2 This is a flowchart illustrating the augmented reality display method provided in an embodiment of this application;
[0062] Figure 3a This is a scene diagram illustrating the augmented reality display method provided in the embodiments of this application;
[0063] Figure 3b This is another scene illustration of the augmented reality display method provided in the embodiments of this application;
[0064] Figure 4 This is another schematic flowchart of the augmented reality display method provided in the embodiments of this application;
[0065] Figure 5a This is another scene illustration of the augmented reality display method provided in the embodiments of this application;
[0066] Figure 5b This is another scene illustration of the augmented reality display method provided in the embodiments of this application;
[0067] Figure 5cThis is another scene illustration of the augmented reality display method provided in the embodiments of this application;
[0068] Figure 5d This is another scene illustration of the augmented reality display method provided in the embodiments of this application;
[0069] Figure 5e This is another scene illustration of the augmented reality display method provided in the embodiments of this application;
[0070] Figure 5f This is a timing diagram of the augmented reality display system provided in the embodiments of this application;
[0071] Figure 6 This is a schematic diagram of the augmented reality display device provided in the embodiments of this application;
[0072] Figure 7 This is a schematic diagram of the terminal structure provided in the embodiments of this application. Detailed Implementation
[0073] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0074] This application provides an augmented reality display method, apparatus, and computer-readable storage medium.
[0075] Please see Figure 1 , Figure 1 This is a schematic diagram of an augmented reality display system provided in an embodiment of this application. It includes a terminal 11, an AR device 12, and a server. The terminal 11 and server 20, as well as the AR device 12 and server, can be connected via a communication network. This communication network includes wireless and wired networks. The wireless network includes one or more combinations of wireless wide area networks (WANs), wireless local area networks (LANs), wireless metropolitan area networks (MANs), and wireless personal networks (PANs). The network includes network entities such as routers and gateways, which are not shown in the diagram. The terminal 11 can interact with the server via the communication network. For example, the server distributes augmented reality data to the terminal 11 and the AR device 12, enabling the terminal 11 and the AR device 12 to display augmented reality data.
[0076] The augmented reality display system may include an augmented reality display device, which may be integrated into a terminal with storage units and a microprocessor, such as a tablet computer, mobile phone, laptop computer, or desktop computer, or into an AR device with storage units and a microprocessor, such as AR glasses or AR headsets. Figure 1 In the middle, this terminal is Figure 1 Terminal 11 in the text refers to the AR device. Figure 1 The AR device 12 in the terminal 11 and AR device 12 can be equipped with various client applications required by users, such as clients with augmented reality functions. The terminal 11 and AR device 12 can be used to display a prompt interface, and display virtual objects, the virtual viewpoint of the virtual objects, and the augmented reality range on the prompt interface; display interactive prompt information on the prompt interface; interactively control the movement of the virtual objects and the orientation of the virtual viewpoint in response to the collected sensor information; and activate the augmented reality display when the interactive state of the virtual objects matches the augmented reality range.
[0077] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms, for storing and updating augmented reality data.
[0078] It should be noted that, Figure 1 The illustrated scene diagram of the augmented reality display system is merely an example. The augmented reality display system and scene described in this application embodiment are for the purpose of more clearly illustrating the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided in this application embodiment. As those skilled in the art will know, with the evolution of augmented reality display systems and the emergence of new business scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems.
[0079] The following sections will provide detailed explanations.
[0080] This application provides an augmented reality display method that can be executed by a client on a terminal.
[0081] It should be noted that while some processes described in the specification, claims, and drawings include multiple steps appearing in a specific order, it should be clearly understood that these steps may not be performed in the order they appear herein, or may be performed in parallel. The step numbers are merely used to distinguish different steps and do not themselves represent any execution order. Furthermore, descriptions such as "first" and "second" in this document are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.
[0082] Please see Figure 2 , Figure 2 This is a flowchart illustrating the augmented reality display method provided in an embodiment of this application. The augmented reality display method includes:
[0083] In step 101, a prompt interface is displayed, showing the virtual object, the virtual viewpoint of the virtual object, and the augmented reality range.
[0084] This application applies augmented reality (AR) technology, a relatively new technology that integrates real-world and virtual-world information. It uses computer technology to simulate and overlay virtual information onto real-world objects that are difficult to experience in the real world, creating a sensory experience that transcends reality. The real environment and virtual objects can coexist in the same scene and space. AR scenes are generally divided into object scenes and scope scenes. For example, in teaching children animal recognition, virtual animals can be overlaid on the real environment to create an augmented reality effect, allowing children to more intuitively identify the animals and their characteristics, significantly improving learning efficiency compared to traditional methods. Similarly, in game scenarios, game items and virtual non-player characters (NPCs) can be overlaid on the real environment, providing an immersive AR experience that greatly enhances the enjoyment of the game compared to traditional methods.
[0085] This can also be understood as placing virtual information in reality and allowing people to interact with it. Therefore, the range of the virtual information displayed (i.e., the location range of the augmented reality display) is defined and limited. In related technologies, if the user's viewing angle does not point to this location range or the user's location is not within this range, the augmented reality display will be paused, with only a simple text prompt, such as "You have left the game area. Please return to the game to continue playing." That is, it only informs the user that they have left the AR location range, and then provides no guidance on how to return to the location range to continue the augmented reality display.
[0086] To address the aforementioned issues, this embodiment of the application provides a visual prompt interface when the augmented reality display is detected to be paused, or when the user has just started the augmented reality display and the user's viewing angle is not pointing towards the specified location range or the user's location is not within that range. The prompt interface displays the augmented reality range corresponding to the location range shown on the prompt interface. It should be noted that the space of this prompt interface can be converted to the space of the real world at a certain ratio, such as a 1:10 ratio (the prompt space is 1, the real space is 10). Therefore, based on the relationship between the user's current location and the specified location range, the corresponding virtual object can be displayed at the appropriate position on the prompt interface. This virtual object can be a virtual character, representing the user's virtual image on the prompt interface.
[0087] Furthermore, the orientation information of the user's current viewing direction can be obtained, which includes at least the viewing angle. Then, a virtual viewpoint of the virtual object can be generated based on the orientation information. The virtual viewpoint represents the range of view that the virtual object can see in the prompt interface, and can be represented by a triangle.
[0088] For a better illustration of the embodiments of this application, please refer to the following: Figure 3a , Figure 3b For reference.
[0089] like Figure 3a As shown, the terminal can display a prompt interface 20, on which the augmented reality range 21, virtual object 22, and virtual viewpoint 23 of the virtual object are displayed.
[0090] In some implementations, displaying a virtual object, the virtual viewpoint of the virtual object, and the augmented reality range on the prompt interface includes:
[0091] (1) Obtain first positioning information, second orientation information, and the location range displayed in augmented reality;
[0092] (2) Generate the corresponding augmented reality range on the prompt interface based on the location range;
[0093] (3) Determine the virtual position of the virtual object on the prompt interface based on the first positioning information;
[0094] (4) Display the virtual object at the virtual location and generate a virtual view of the virtual object based on the second orientation information.
[0095] Specifically, the first positioning information can be obtained through the gyroscope on the terminal. This positioning information can be Global Positioning System (GPS) information. The second orientation information is determined by the orientation of the camera or the gyroscope. That is, the first positioning information is used to indicate the current position of the terminal, and the second orientation information is used to indicate the current orientation of the terminal. Finally, the location range of the augmented reality display is determined according to the positioning set indicated by the background.
[0096] Furthermore, according to a preset ratio between the area of the prompt interface and the real environment, such as a 1:10 ratio, the corresponding augmented reality range of the location range can be generated on the prompt interface, and the virtual position of the virtual object on the prompt interface can be found according to the first positioning information and the preset ratio.
[0097] The virtual object is displayed at a virtual location on the prompt interface, and the virtual view of the virtual object on the prompt interface is restored according to the orientation of the second orientation information. In this way, according to a preset ratio, the user, the user's view, and the augmented reality location range are restored on the prompt interface to form the corresponding virtual object, the virtual view of the virtual object, and the augmented reality range, so as to prompt the user in a visual form.
[0098] In step 102, interactive prompts are displayed on the prompt interface.
[0099] Interactive prompts can be displayed on the prompt interface. These prompts indicate that the user's position or viewpoint is outside the augmented reality display area, instructing the user to move or turn back to the augmented reality display area. For example, the prompts could be "Please move your viewpoint back to the game area to start the game" or "Please move back to the game area to start the game." These examples are not intended to be specific limitations.
[0100] In some implementations, following the display prompt interface, the following further includes:
[0101] (1.1) Generate a vibration command and generate vibration feedback based on the vibration command.
[0102] In order to remind the user that the augmented reality display is paused, a vibration command can be generated and executed to generate vibration feedback to remind the user that the augmented reality display is paused. The vibration feedback time can be a preset time, such as 5 seconds. The vibration feedback will only stop when the vibration feedback time reaches 5 seconds or when the augmented reality display is restarted.
[0103] In some implementations, displaying interactive prompt information on the prompt interface includes at least one of the following steps:
[0104] (2.1) Display a text box on the prompt interface and display interactive prompt information in the text box.
[0105] (2.2) Play interactive prompts by voice and display the corresponding images on the prompt interface.
[0106] This prompt interface can display a text box with interactive prompts to guide the user on how to reactivate the augmented reality display. The interactive prompts will vary depending on the augmented reality scenario; please refer to the following documentation. Figure 3a For object-oriented scenes, the augmented reality display condition is the user's viewpoint within the augmented reality display area. Therefore, the interactive prompt message can be "Please move your viewpoint back to the game area to start the game." For range-oriented scenes, the augmented reality display condition is the user's location within the augmented reality display area. Therefore, the interactive prompt message can be "Please move back to the game area to start the game."
[0107] Interactive prompts can also be played via voice. For example, the system can preset a voice prompt such as "Please move your view back to the game area to start the game." After hearing the voice prompt, users can intuitively understand how to restart AR. To achieve a better prompting effect, the corresponding image generated by the interactive prompt can also be displayed on the prompt interface. This image is a picture, which can be an image control. Clicking the image control will quickly repeat the voice prompt once. This click can be a single click, double click, or long press.
[0108] In step 103, in response to the collected sensor information, interactive control is performed on the movement of virtual objects and the orientation of the virtual viewpoint in the prompt interface.
[0109] The response stated in the embodiments of this application is used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which the operation depends are met, one or more operations performed may be performed in real time or may have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.
[0110] Among them, the corresponding augmented reality display conditions are different for different augmented reality display scenarios. For object scenarios, the augmented reality display conditions can be that the user's viewpoint is within the location range of the augmented reality display, that is, the observation of AR objects can be realized. For range scenarios, the augmented reality display conditions can be that the user is located within the location range of the augmented reality display.
[0111] In this way, the interaction between the real user and the virtual object in the prompt interface can be established. That is, by collecting sensor information in the real world, the movement of the virtual object in the prompt interface and the orientation of the virtual view can be interactively controlled. In other words, the real user can change the positioning of the gyroscope and the orientation information of the terminal by moving, and control the virtual object in the prompt interface to move and rotate according to the corresponding preset ratio.
[0112] It is understood that in the specific embodiments of this application, data such as user information and sensor information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0113] In some implementations, the sensor information includes at least displacement data and steering data. The interactive control of the movement of the virtual object and the orientation of the virtual viewpoint in response to the acquired sensor information includes:
[0114] (1) In response to the displacement data in the sensor information, interactive control is performed on the movement of the virtual object;
[0115] (2) In response to the steering data in the sensor information, interactively control the orientation of the virtual viewpoint.
[0116] The sensor information may include at least displacement data and steering data. The displacement data indicates the direction and distance the user moves relative to a previous fixed point, and the steering data indicates the angle of rotation of the user's orientation relative to the previous orientation.
[0117] Based on this, in response to the displacement data in the sensor information, the virtual object in the prompt interface is moved according to a preset ratio, and in response to the turning data in the sensor information, the orientation of the virtual view of the virtual object is rotated. In this way, communication is achieved, and users can accurately adjust the position and direction in the real world through the reference of the prompt interface.
[0118] In some implementations, after interactively controlling the movement of the virtual object, the method further includes:
[0119] (1.1) Display the first identifier on the prompt interface.
[0120] In order to ensure the effect of subsequent augmented reality display, the virtual position of the virtual object needs to be as close as possible to the augmented reality range (that is, the closer the user is to the augmented reality display range). The distance can be calculated by calculating the distance between the edge of the augmented reality range and the position of the virtual object. The edge can refer to the edge point that forms the smallest straight-line distance between the virtual object and the virtual object.
[0121] In another implementation, calculating the distance between the virtual position of the virtual object and the augmented reality range can be done by calculating the distance between the virtual position of the virtual object and the geometric center point of the augmented reality range. Assuming the augmented reality range is circular, the geometric center point is the center of the circle.
[0122] Therefore, after interactively controlling the movement of virtual objects using displacement data, it is necessary to detect whether the distance between the virtual position of the virtual object and the edge of the augmented reality range has increased.
[0123] Furthermore, when the distance between the virtual position of the virtual object and the edge of the augmented reality range is detected to increase, it indicates that the virtual position of the virtual object is getting further and further away from the augmented reality range. In order to remind the user, the first indicator "!" can be displayed on the prompt interface, and corresponding text can be displayed, such as "Note that it is getting further and further away from the augmented reality range, please adjust in time". That is, the first indicator is used to prompt the virtual object to move away from the augmented reality range.
[0124] If the distance between the virtual position of the virtual object and the edge of the augmented reality range is not increased, it means that the virtual position of the virtual object is not far from the augmented reality range, and the first identifier does not need to be displayed.
[0125] In some implementations, it also includes:
[0126] (2.1) Displacement data is collected using a gyroscope;
[0127] (2.2) Obtain the first orientation information of the camera, and generate turning data based on the first orientation information;
[0128] (2.3) Determine sensor information based on the displacement and steering data.
[0129] Displacement data can be collected through the gyroscope on the terminal. This displacement data represents the direction and distance the user moves relative to the previous fixed point, and can be used to calculate the distance difference between the previous fixed point and the current positioning point.
[0130] Furthermore, this first orientation information represents the orientation corresponding to the user's current observation direction. In other words, the first orientation information indicates the terminal's current orientation, and the unit can be calculated in degrees. In some embodiments, this first orientation information can also be collected by a gyroscope. Since this application embodiment is applied to an augmented reality scenario, and augmented reality is achieved by overlaying virtual information onto images of the real environment captured by the terminal's camera, the camera's orientation is crucial for realizing augmented reality display. Therefore, the camera's first orientation information can be obtained as the user's current orientation information, and turning data can be generated based on this first orientation information. This turning data indicates the angle of rotation of the user's orientation relative to the previous orientation, which can be calculated from the angle difference between the previous orientation and the current orientation. Finally, this displacement data and turning data are used as sensor information to control the movement and turning of virtual objects through sensor information.
[0131] In step 104, when the interaction state of the virtual object is detected to match the augmented reality range, the augmented reality display is turned on.
[0132] The interactive state of the virtual object can include its virtual location state and its virtual viewing angle state. Different augmented reality (AR) display scenarios have different AR display conditions. For object scenarios, the AR display condition can be that the user's viewing angle is within the AR display's location range. This is reflected in the prompt interface as the virtual object's virtual viewing angle is within the AR range, enabling object observation. For range scenarios, the AR display condition can be that the user is positioned within the AR display's location range. This is reflected in the prompt interface as the virtual object's virtual location is within the AR range.
[0133] Therefore, based on different augmented reality display scenarios, it is possible to detect whether the interaction state of a virtual object matches the augmented reality range. For object scenarios, when it is detected that the virtual viewpoint of the virtual object is facing the augmented reality range, the interaction state of the virtual object matches the augmented reality range, and the augmented reality display can be activated. For range scenarios, when it is detected that the virtual position of the virtual object has moved into the augmented reality range, the interaction state of the virtual object matches the augmented reality range, and the augmented reality display can be activated. This allows users to receive visual guidance on activating the augmented reality display through a prompt interface. Compared to simple text prompts, this embodiment can efficiently activate the augmented reality display and guide users to smoothly re-enter the augmented reality display.
[0134] In some implementations, the method further includes:
[0135] (1) Obtain the current third orientation information and the location range of the augmented reality display;
[0136] (2) When it is detected that the orientation indicated by the third orientation information does not point to the location range, the augmented reality display is paused and a prompt interface is displayed.
[0137] For example, regarding object-oriented scenarios, please refer to the following: Figure 3a As shown, the current third orientation information of the terminal's camera and the position range of the enhanced display can be obtained. This third orientation information can represent the orientation of the user's current viewpoint, that is, this third orientation information is used to indicate the orientation of the terminal.
[0138] For an object scene, the augmented reality display condition is that the user's viewing angle is within the augmented reality display location range. Therefore, during the augmented reality display process, it is necessary to detect in real time whether the orientation indicated by the third orientation information points to this location range. When it is detected that the orientation indicated by the third orientation information points to this location range, the augmented reality display can continue. However, when it is detected that the orientation indicated by the third orientation information does not point to this location range, the augmented reality display needs to be paused, and the steps of displaying the prompt interface 20 need to be executed. In the prompt interface 20, the virtual viewpoint 23 of the virtual object 22 is not facing the augmented reality range 21.
[0139] In some implementations, when the interaction state of a virtual object is detected to match the augmented reality range, the augmented reality display is activated, including:
[0140] (1.1) When the virtual view of the virtual object is detected to be facing the augmented reality range, the augmented reality display is activated.
[0141] Specifically, when the virtual viewpoint of the virtual object is detected to be facing the augmented reality range, which represents the range of the user's current viewpoint in the real environment, the conditions for augmented reality display are met, and augmented reality display continues to be enabled.
[0142] In one embodiment, to avoid repeatedly switching augmented reality displays, the augmented reality display can be activated when the time when the virtual viewpoint of the detected virtual object is facing the augmented display range reaches a preset time, such as 3 seconds.
[0143] In some implementations, after displaying the prompt interface, the following is also included:
[0144] (2.1) Generate corresponding turning arrows and turning prompts based on the relationship between the virtual viewpoint's orientation and the augmented reality range;
[0145] (2.2) Display the turning arrow on the virtual object and announce the turning prompt via voice.
[0146] Please refer to the following: Figure 3b As shown, to better guide users on how to activate augmented reality, corresponding turning arrows and cues can be calculated based on the relationship between the virtual viewpoint's orientation and the augmented reality's range, such as... Figure 3b As shown, the virtual viewpoint 23 of the virtual object 22 needs to be rotated 180 degrees counterclockwise to reach the augmented reality range. Therefore, a counterclockwise 180-degree turning arrow and a turning prompt "counterclockwise 180 degrees" can be generated, and the counterclockwise 180-degree turning arrow can be displayed on the virtual object and the turning prompt can be broadcast by voice. In this way, the user can quickly know that the viewpoint has been rotated 180 degrees counterclockwise and realize the activation of augmented reality display.
[0147] As described above, the embodiments of this application display a prompt interface, on which virtual objects, virtual viewpoints of the virtual objects, and augmented reality ranges are displayed. Interactive prompt information is displayed on this interface. In response to collected sensor information, interactive control is performed on the movement of the virtual objects and the orientation of the virtual viewpoints within the prompt interface. When the interactive state of the virtual object matches the augmented reality range, the augmented reality display is activated. Thus, the virtual objects, virtual viewpoints, and augmented reality ranges are displayed in a visual form on the prompt interface, allowing users to move and turn based on the prompt interface, and simultaneously interactively control the movement and orientation of the virtual objects. This achieves visual guidance. When the interactive state of the virtual object matches the augmented reality range, the augmented reality display is quickly activated. Compared to solutions that only provide simple text prompts to guide users back to the AR range, users can intuitively understand how to activate the augmented reality display and operate efficiently based on visual guidance, greatly improving the efficiency of the augmented reality display.
[0148] In this embodiment, the augmented reality display device will be specifically integrated into the terminal as an example for explanation. Please refer to the following description for details.
[0149] Please refer to Figure 3, which is another schematic flowchart of the augmented reality display method provided in an embodiment of this application. The method flow may include:
[0150] In step 201, the terminal obtains the current second location information and the location range displayed by augmented reality.
[0151] In this application embodiment, an example of augmented reality display in a range scene is given. The second positioning information of the terminal's gyroscope and the location range of the augmented reality display can be obtained. The second positioning information represents the current user's positioning point (which can also be understood as the terminal's position), and the location range of the augmented reality display is a set of positioning points of the augmented reality display.
[0152] For a better illustration of the embodiments of this application, please refer to the following: Figure 5a , Figure 5b , Figure 5c and Figure 5d For reference.
[0153] like Figure 5a As shown, the rectangular area is the location range of the augmented reality display. When the location point indicated by the current second location information is detected within the location range of the augmented reality display, the augmented reality display is in normal state, that is, the AR environment is presented on the user's terminal, and the user can have an AR experience within this location range.
[0154] In step 202, when the terminal detects that the location indicated by the second location information is not within the location range, the augmented reality display is paused.
[0155] In range scenarios, augmented reality display conditions can be defined as the user being positioned within the augmented reality display's location range, such as... Figure 5b As shown, when the terminal detects that the location indicated by the second location information is not within the range of the location, it means that the user has walked outside the range of the location and does not meet the conditions for augmented reality display, so the augmented reality display can be paused.
[0156] In step 203, the terminal displays a prompt interface, obtains the first positioning information, the second orientation information, and the location range of the augmented reality display, and generates the corresponding augmented reality range on the prompt interface according to the location range.
[0157] Among them, such as Figure 5cAs shown, the terminal can display a prompt interface 30, obtain the current first positioning information (i.e., the terminal's current location information) through the gyroscope on the terminal. This positioning information can be GPS information. The second orientation information (i.e., the terminal's current orientation) is determined by the orientation of the camera. Finally, the location range of the augmented reality display is determined according to the GPS positioning set indicated by the background.
[0158] Furthermore, according to a preset ratio between the area of the prompt interface and the real environment, such as a 1:8 ratio, the corresponding augmented reality range 31 can be generated on the prompt interface for this location range.
[0159] In step 204, the terminal determines the virtual position of the virtual object on the prompt interface based on the first positioning information, displays the virtual object at the virtual position, and generates a virtual view of the virtual object based on the second orientation information.
[0160] Specifically, based on the first positioning information and a preset ratio, the virtual position of the virtual object on the prompt interface is found, and the virtual object 32 is displayed at the virtual position on the prompt interface. Furthermore, based on the orientation of the second orientation information, the virtual viewpoint 33 of the virtual object 32 on the prompt interface is restored. The orientation of the virtual viewpoint 33 is determined based on the second orientation information. Thus, according to the preset ratio, the user, the user's viewpoint, and the augmented reality position range are restored on the prompt interface to form the corresponding virtual object 32, the virtual viewpoint 33 of the virtual object, and the augmented reality range 31, which are then displayed to the user in a visual form.
[0161] In step 205, the terminal generates vibration data and generates vibration feedback of corresponding intensity based on the vibration data. A text box is displayed on the prompt interface, and interactive prompt information is displayed in the text box.
[0162] Since the augmented reality display is paused, vibration data can be generated to remind the user that the display is paused. The larger the vibration data, the greater the vibration amplitude of the terminal's motor; the smaller the vibration data, the smaller the vibration amplitude of the terminal's motor.
[0163] Since the augmented reality display condition in this application embodiment is that the user's location information returns to the location range, the magnitude of the vibration data can be set to be proportional to the distance between the location indicated by the second location information and the edge of the location range. That is, the greater the distance between the location indicated by the second location information and the edge of the location range, the greater the vibration data; the smaller the distance between the location indicated by the second location information and the edge of the location range, the smaller the vibration data. The edge of the location range can be the edge point on the edge of the location range with the shortest straight-line distance from the second location information.
[0164] Therefore, by generating vibration feedback of corresponding intensity based on the relevant vibration data, the user can be better reminded of the relationship between the current location and the surrounding area, thus providing flexible prompts.
[0165] Furthermore, a text box can be displayed on the prompt screen, with interactive prompts indicating how to re-enable the augmented reality display. See also... Figure 5c The interactive prompt message could be "Please move back to the game area to start the game".
[0166] In step 206, the terminal generates a corresponding navigation trajectory and navigation prompts based on the virtual position of the virtual object, the orientation of the virtual viewpoint, and the relationship with the augmented reality range. The navigation trajectory is displayed on the virtual object, and the navigation prompts are broadcast via voice.
[0167] To better guide users on how to enable augmented reality (AR) display, the terminal can calculate the corresponding navigation trajectory and prompts based on the relationship between the virtual position of virtual objects, the orientation of the virtual viewpoint, and the AR range. Please refer to the documentation for further details. Figure 5d As shown, the virtual viewpoint 23 of the virtual object faces the upper right, and the straight-line distance from the virtual object to the edge of the augmented reality range is 2 meters. Therefore, a navigation trajectory can be generated by taking the virtual object as the starting point and the edge point on the edge of the augmented reality range that is closest to the virtual object in a straight line as the ending point. The navigation trajectory can display the distance and direction of the trajectory, and can also generate a navigation prompt "move 2 meters to the lower right". In this way, the navigation trajectory can be displayed on the virtual object, and the turning prompt can be broadcast by voice. In this way, the user can quickly know to move 2 meters to the lower right and realize the augmented reality display.
[0168] In step 207, the terminal collects displacement data through the gyroscope, obtains the first orientation information of the camera, generates steering data based on the first orientation information, and determines sensor information based on the displacement data and steering data.
[0169] This allows for the establishment of an interactive relationship between real users and virtual objects in the prompt interface. Specifically, sensor information collected in the real world can be used to interactively control the movement of virtual objects and the orientation of the virtual viewpoint in the prompt interface. This involves collecting displacement data through the gyroscope on the terminal, which represents the direction and distance of the user's movement relative to a previous fixed point. This displacement data can be calculated from the distance difference between the previous fixed point and the current positioning point.
[0170] Furthermore, this first orientation information represents the orientation corresponding to the user's current observation direction, and the unit can be calculated in angles. In this embodiment, the first orientation information of the camera is obtained as the user's current first orientation information, and turning data is generated based on this first orientation information. This turning data indicates the angle of rotation of the user's orientation relative to the previous orientation, which can be calculated from the angle difference between the previous orientation and the current orientation. Finally, the displacement data and turning data are used as sensor information to realize the control of the movement and turning of the virtual object through sensor information.
[0171] In step 208, the terminal responds to the displacement data in the sensor information to interactively control the movement of the virtual object. When it detects that the distance between the virtual position of the virtual object and the edge of the augmented reality range increases, a first identifier is displayed on the prompt interface. In response to the turning data in the sensor information, the terminal interactively controls the orientation of the virtual viewpoint.
[0172] In response to the displacement data in the sensor information, the virtual object in the prompt interface is moved according to a preset ratio. In response to the turning data in the sensor information, the virtual viewpoint of the virtual object is rotated. In this way, communication is achieved. Users can accurately adjust the position and turning in the real world through the reference of the prompt interface until they move into the augmented reality range.
[0173] Furthermore, to avoid increasing the difficulty of augmented reality display due to the increased distance between the virtual position of the virtual object and the augmented reality range, it is necessary to detect whether the distance between the virtual position of the virtual object and the edge of the augmented reality range has increased after interactively controlling the movement of the virtual object through displacement data.
[0174] The distance between the virtual location and the edge of the augmented reality range can be calculated as the minimum distance between the virtual location and the edge of the augmented reality range. When an increase in the distance between the virtual location and the edge of the augmented reality range is detected, it indicates that the virtual location of the virtual object is getting further and further away from the augmented reality range. To remind the user, the first indicator "!" can be displayed on the prompt interface, along with corresponding text, such as "Note that you are getting further and further away from the augmented reality range. Please adjust accordingly."
[0175] In step 209, when the terminal detects that the virtual position of the virtual object has shifted into the augmented reality range, the augmented reality display is activated.
[0176] Specifically, when the terminal detects that the virtual position of a virtual object has shifted into the augmented reality range, it means that the user's current location is within the range in the real environment. Therefore, the conditions for augmented reality display are met, and augmented reality display continues to be enabled.
[0177] In one embodiment, to avoid repeatedly switching augmented reality displays, the terminal can determine that the user's current position is stable within the augmented reality range when the time it detects the virtual object's virtual position displacement within the augmented reality range reaches a preset time, such as 2 seconds. In this state, the augmented reality display is then turned on.
[0178] As described above, this embodiment of the application displays a prompt interface, showing a virtual object, the virtual viewpoint of the virtual object, and the augmented reality range on the prompt interface; it also displays interactive prompt information on the prompt interface; in response to collected sensor information, it interactively controls the movement of the virtual object and the orientation of the virtual viewpoint in the prompt interface; and when the interactive state of the virtual object matches the augmented reality range, it activates the augmented reality display. In this way, the virtual object, virtual viewpoint, and augmented reality range are displayed in a visual form on the prompt interface, allowing the user to move and turn based on the prompt interface, and simultaneously interactively control the movement and orientation of the virtual object, achieving visual guidance. When the interactive state of the virtual object matches the augmented reality range, the augmented reality display is quickly activated. Compared to solutions that only provide simple text prompts to guide the user back to the AR range, users can intuitively understand how to activate the augmented reality display and operate efficiently based on visual guidance, greatly improving the efficiency of the augmented reality display.
[0179] Furthermore, navigation and voice prompts can be used to remind users how to interact and control the system, and to enable augmented reality display, thus improving the efficiency of augmented reality display.
[0180] For a better description of the embodiments of this application, please refer to the following: Figure 5e and Figure 5f As shown, Figure 5e This is another schematic diagram of a scenario for the augmented reality display method provided in the embodiments of this application. Figure 5f This is a timing diagram of the augmented reality display system provided in the embodiments of this application.
[0181] like Figure 5e As shown, when a user leaves the AR range during an AR experience, the AR game will pause. The device (i.e., the terminal) will provide vibration feedback and display a prompt interface to show the player's orientation and position, as well as the location range constructed by AR. The player can use the prompt interface and guidance to move and control their orientation to return to the location range and resume the normal AR game.
[0182] like Figure 5f As shown, the AR backend can be the terminal's backend, the AR frontend is the content currently displayed on the screen, and the user operation is the player's operation.
[0183] Thus, 1. The AR backend can construct the range and location of the AR game; 2. The AR frontend displays the AR game model and related UI (user interface, i.e., augmented reality interface); 3. The user holds the device and plays within the range; 4. When the user accidentally moves out of the AR range; 5. The AR backend detects the status and enters the pause page; 6. The gyroscope emits a vibration alert, and the AR frontend displays the user's position and the AR's position in real time, guiding the player back into the AR range; 7. The user moves back into the AR range; 8. The AR backend returns to normal, and the pause is canceled; 9. The AR frontend resumes normal AR game playback.
[0184] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the augmented reality display method above, which will not be repeated here.
[0185] To facilitate better implementation of the augmented reality display method provided in the embodiments of this application, the embodiments of this application also provide an apparatus based on the above-described augmented reality display method. The meanings of the terms used are the same as in the above-described augmented reality display method, and specific implementation details can be found in the descriptions in the method embodiments.
[0186] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an augmented reality display device provided in an embodiment of this application. The augmented reality display device may include a first display unit 401, a second display unit 402, a control unit 403, and an activation unit 404, etc.
[0187] The first display unit 401 is used to display a prompt interface and to display a virtual object, the virtual viewpoint of the virtual object, and the augmented reality range on the prompt interface.
[0188] In some embodiments, the first display unit 401 is further configured to:
[0189] Display a prompt interface;
[0190] Obtain the first location information, the second orientation information, and the location range displayed in augmented reality;
[0191] Wherein, the first positioning information is used to indicate the location of the terminal, and the second orientation information is used to indicate the orientation of the terminal;
[0192] Based on the location range, a corresponding augmented reality range is generated on the prompt interface;
[0193] Based on the first location information, the virtual position of the virtual object on the prompt interface is determined;
[0194] The virtual object is displayed at the virtual location, and a virtual view of the virtual object is generated based on the second orientation information.
[0195] In some embodiments, the device further includes an identification display unit for:
[0196] In some embodiments, the device further includes an identification display unit for:
[0197] The first identifier is displayed on this prompt screen;
[0198] The first identifier is used to indicate that the virtual object is far from the augmented reality range.
[0199] In some embodiments, the device further includes a feedback prompting unit for:
[0200] Generate a vibration command, and generate vibration feedback based on the vibration command.
[0201] The second display unit 402 is used to display interactive prompt information on the prompt interface.
[0202] The second display unit 402 is used for at least one of the following steps:
[0203] Display a text box on the prompt interface and display interactive prompt information in the text box;
[0204] Interactive prompts are played via voice, and corresponding images are displayed on the prompt interface.
[0205] The control unit 403 is used to interactively control the movement of the virtual object and the orientation of the virtual viewpoint in the prompt interface in response to the collected sensor information.
[0206] In some embodiments, the sensor information includes at least displacement data and steering data, and the control unit 403 is configured to:
[0207] In response to the displacement data in the sensor information, the movement of the virtual object is interactively controlled.
[0208] In response to the steering data in the sensor information, the orientation of the virtual viewpoint is interactively controlled.
[0209] In some embodiments, the apparatus further includes a determining unit for:
[0210] Displacement data is collected using a gyroscope;
[0211] Acquire first orientation information of the camera, and generate turning data based on the first orientation information, wherein the first orientation information is used to represent the orientation of the terminal;
[0212] The sensor information is determined based on the displacement and steering data.
[0213] The activation unit 404 is used to activate the augmented reality display when the interaction state of the virtual object is detected to match the augmented reality range.
[0214] In some embodiments, the device further includes a first pause unit for:
[0215] Obtain the current second location information and the location range displayed by augmented reality, the second location information being used to indicate the location of the terminal;
[0216] When the location indicated by the second location information is detected to be outside the specified range, the augmented reality display is paused and a prompt message is displayed.
[0217] The opening unit 404 is used for;
[0218] When the virtual position of the virtual object is detected to have shifted into the augmented reality range, the augmented reality display is activated.
[0219] In some embodiments, the device further includes a first prompting unit for:
[0220] Based on the virtual object's virtual location, the orientation of the virtual viewpoint, and the relationship with the augmented reality range, a corresponding navigation trajectory and navigation prompts are generated;
[0221] The navigation path is displayed on the virtual object, and the navigation prompts are read aloud via voice.
[0222] In some embodiments, the device further includes a vibration generating unit for:
[0223] Generate vibration data, and generate vibration feedback of corresponding intensity based on the vibration data;
[0224] The magnitude of the vibration data is proportional to the distance between the location indicated by the second positioning information and the location range.
[0225] In some embodiments, the device further includes a second pause unit for:
[0226] Obtain the current third orientation information and the location range of the augmented reality display, which is used to indicate the orientation of the terminal;
[0227] When the orientation indicated by the third orientation information is detected to be outside the specified location range, the augmented reality display is paused, and a prompt message is displayed.
[0228] The opening unit 404 is also used for;
[0229] When the virtual viewpoint of the virtual object is detected to be facing the augmented reality range, the augmented reality display is activated.
[0230] In some embodiments, the device further includes a second prompting unit for:
[0231] Based on the relationship between the virtual viewpoint's orientation and the augmented reality range, generate corresponding turning arrows and turning prompts;
[0232] Display the turn arrow on the virtual object and announce the turn prompt via voice.
[0233] The specific implementation of each of the above units can be found in the previous embodiments, and will not be repeated here.
[0234] This application also provides a computer device, which can be a terminal, such as... Figure 7 As shown, it illustrates the structural diagram of the terminal involved in the embodiments of this application, specifically:
[0235] The computer device may include radio frequency (RF) circuitry 501, a memory 502 including one or more computer-readable storage media, an input unit 503, a display unit 504, a sensor 505, audio circuitry 506, a wireless fidelity (WiFi) module 507, a processor 508 including one or more processing cores, and a power supply 509, among other components. Those skilled in the art will understand that... Figure 7 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:
[0236] RF circuit 501 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and hands it over to one or more processors 508 for processing; additionally, it transmits uplink data to the base station. Typically, RF circuit 501 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 501 can also communicate wirelessly with networks and other devices. Wireless communication can use any communication standard or protocol, including but not limited to GSM, GPRS, CDMA, WCDMA, LTE, email, and SMS.
[0237] The memory 502 can be used to store software programs and modules. The processor 508 executes various functional applications and augmented reality displays by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the terminal (such as audio data, phone book, etc.). In addition, the memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide access to the memory 502 for the processor 508 and the input unit 503.
[0238] Input unit 503 can be used to receive input numerical or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, in one embodiment, input unit 503 may include a touch-sensitive surface and other input devices. A touch-sensitive surface, also known as a touch display or touchpad, can collect user touch operations on or near it (e.g., user operations using fingers, styluses, or any suitable object or accessory on or near the touch-sensitive surface) and drive corresponding connection devices according to a pre-set program. Optionally, the touch-sensitive surface may include a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 508, and can receive and execute commands from the processor 508. Furthermore, various types of touch-sensitive surfaces, such as resistive, capacitive, infrared, and surface acoustic wave, can be used. In addition to the touch-sensitive surface, input unit 503 may also include other input devices. Specifically, other input devices may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.
[0239] Display unit 504 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the terminal. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Display unit 504 may include a display panel, optionally configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar form. Furthermore, a touch-sensitive surface may cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it transmits the information to processor 508 to determine the type of touch event. Subsequently, processor 508 provides corresponding visual output on the display panel according to the type of touch event. Although in Figure 7 In this context, the touch-sensitive surface and the display panel are two separate components for implementing input and output functions. However, in some embodiments, the touch-sensitive surface and the display panel can be integrated to achieve both input and output functions.
[0240] The terminal may also include at least one sensor 505, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel according to the ambient light level, and the proximity sensor can turn off the display panel and / or backlight when the terminal is moved to the ear. As a type of motion sensor, a gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that the terminal may also be equipped with, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.
[0241] Audio circuitry 506, a speaker, and a microphone provide an audio interface between the user and the terminal. Audio circuitry 506 converts received audio data into electrical signals, transmits them to the speaker, and the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuitry 506, converted back into audio data, and processed by processor 508. The processed data is then transmitted via RF circuitry 501 to, for example, another terminal, or output to memory 502 for further processing. Audio circuitry 506 may also include an earphone jack to facilitate communication between a peripheral headset and the terminal.
[0242] WiFi is a short-range wireless transmission technology. Terminals using the WiFi module 507 can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 7 WiFi module 507 is shown, but it is understood that it is not a necessary component of the terminal and can be omitted as needed without changing the essence of the invention.
[0243] The processor 508 is the control center of the terminal, connecting various parts of the phone via various interfaces and lines. It executes software programs and / or modules stored in the memory 502, and calls data stored in the memory 502 to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 508 may include one or more processing cores; preferably, the processor 508 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 508.
[0244] The terminal also includes a power supply 509 (such as a battery) to power various components. Preferably, the power supply can be logically connected to the processor 508 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 509 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0245] Although not shown, the terminal may also include a camera, Bluetooth module, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 508 in the terminal will load the executable files corresponding to the processes of one or more applications into the memory 502 according to the following instructions, and the processor 508 will run the applications stored in the memory 502 to realize various functions:
[0246] Display a prompt interface, and on this prompt interface, display the virtual object, the virtual viewpoint of the virtual object, and the augmented reality range;
[0247] Display interactive prompts on this prompt interface;
[0248] In response to the collected sensor information, the movement of the virtual object and the orientation of the virtual viewpoint in the prompt interface are interactively controlled;
[0249] When the interaction state of the virtual object is detected to match the augmented reality range, the augmented reality display is turned on.
[0250] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the augmented reality display method above, which will not be repeated here.
[0251] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.
[0252] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the augmented reality display methods provided in embodiments of this application. For example, the instructions can execute the following steps:
[0253] Display a prompt interface, and on this prompt interface, display the virtual object, the virtual viewpoint of the virtual object, and the augmented reality range;
[0254] Display interactive prompts on this prompt interface;
[0255] In response to the collected sensor information, the movement of the virtual object and the orientation of the virtual viewpoint in the prompt interface are interactively controlled;
[0256] When the interaction state of the virtual object is detected to match the augmented reality range, the augmented reality display is turned on.
[0257] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.
[0258] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.
[0259] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0260] Since the instructions stored in the computer-readable storage medium can execute the steps of any of the augmented reality display methods provided in the embodiments of this application, the beneficial effects that any of the augmented reality display methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.
[0261] The above provides a detailed description of an augmented reality display method, apparatus, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. An augmented reality display method, characterized in that, The method includes: A prompt interface is displayed, and a virtual object, the virtual viewpoint of the virtual object, and the augmented reality range are displayed on the prompt interface; wherein, the prompt interface is displayed when at least one of the following occurs: the viewing angle of the user represented by the virtual object does not point to the location range of the augmented reality display, the user's location is not within the location range of the augmented reality display, and the augmented reality display is paused; Display interactive prompts on the prompt interface; In response to the collected sensor information, the movement of the virtual object in the prompt interface and the orientation of the virtual viewpoint are interactively controlled; When the interaction state of the virtual object is detected to match the augmented reality range, the augmented reality display is enabled.
2. The augmented reality display method according to claim 1, characterized in that, The sensor information includes at least displacement data and steering data. The interactive control of the movement of the virtual object and the orientation of the virtual viewpoint in response to the collected sensor information includes: In response to displacement data in the sensor information, interactive control is performed on the movement of the virtual object; In response to the steering data in the sensor information, the orientation of the virtual viewpoint is interactively controlled.
3. The augmented reality display method according to claim 2, characterized in that, After interactively controlling the movement of the virtual object, the method further includes: The first identifier is displayed on the prompt interface; The first identifier is used to indicate that the virtual object is far from the augmented reality range.
4. The augmented reality display method according to claim 2 or 3, characterized in that, The method further includes: Displacement data is collected using a gyroscope; Acquire the first orientation information of the camera, and generate turning data based on the first orientation information, wherein the first orientation information is used to indicate the orientation of the terminal; Sensor information is determined based on the displacement and steering data.
5. The augmented reality display method according to claim 1, characterized in that, The display of the virtual object, the virtual viewpoint of the virtual object, and the augmented reality range on the prompt interface includes: Obtain first location information, second orientation information, and the location range displayed in augmented reality; Wherein, the first positioning information is used to indicate the location of the terminal, and the second orientation information is used to indicate the orientation of the terminal; Based on the location range, a corresponding augmented reality range is generated on the prompt interface; The virtual position of the virtual object on the prompt interface is determined based on the first positioning information; The virtual object is displayed at the virtual location, and a virtual view of the virtual object is generated based on the second orientation information.
6. The augmented reality display method according to claim 1, characterized in that, Following the display prompt interface, it also includes Generate vibration commands and generate vibration feedback based on the vibration commands; Displaying interactive prompt information on the prompt interface includes at least one of the following steps: A text box is displayed on the prompt interface, and interactive prompt information is displayed in the text box; Interactive prompts are played via voice, and corresponding images are displayed on the prompt interface.
7. The augmented reality display method according to claim 1, characterized in that, The method further includes: Obtain the current second location information and the location range displayed by augmented reality, wherein the second location information is used to indicate the location of the terminal; When it is detected that the location indicated by the second location information is not within the specified location range, the augmented reality display is paused and a prompt interface is displayed. The step of activating augmented reality display when the interaction state of the virtual object is detected to match the augmented reality range includes: When the virtual position of the virtual object is detected to have shifted into the augmented reality range, the augmented reality display is activated.
8. The augmented reality display method according to claim 7, characterized in that, Following the display prompt interface, the system also includes: Based on the virtual position of the virtual object, the orientation of the virtual viewpoint, and the relationship with the augmented reality range, a corresponding navigation trajectory and navigation prompts are generated; The navigation trajectory is displayed on the virtual object, and the navigation prompts are read aloud via voice.
9. The augmented reality display method according to claim 7, characterized in that, Following the display prompt interface, the system also includes: Vibration data is generated, and vibration feedback of corresponding intensity is generated based on the vibration data; The magnitude of the vibration data is proportional to the distance between the location indicated by the second positioning information and the location range.
10. The augmented reality display method according to claim 1, characterized in that, The method further includes: Obtain the current third orientation information and the location range of the augmented reality display, wherein the third orientation information is used to indicate the orientation of the terminal; When it is detected that the orientation indicated by the third orientation information does not point to the location range, the augmented reality display is paused and a prompt interface is displayed. The step of activating augmented reality display when the interaction state of the virtual object is detected to match the augmented reality range includes: When the virtual viewpoint of the virtual object is detected to be facing the augmented reality range, the augmented reality display is activated.
11. The augmented reality display method according to claim 10, characterized in that, Following the display prompt interface, the system also includes: Based on the relationship between the orientation of the virtual viewpoint and the augmented reality range, generate corresponding turning arrows and turning prompts; The turning arrow is displayed on the virtual object, and the turning prompt is announced via voice.
12. An augmented reality display device, characterized in that, include: The first display unit is used to display a prompt interface, and to display a virtual object, the virtual viewpoint of the virtual object, and the augmented reality range on the prompt interface; The prompt interface is displayed when at least one of the following occurs: the user's viewing angle represented by the virtual object is not pointing towards the location range of the augmented reality display; the user's location is not within the location range of the augmented reality display; or the augmented reality display is paused. The second display unit is used to display interactive prompt information on the prompt interface; A control unit is used to interactively control the movement of the virtual object and the orientation of the virtual viewpoint in the prompt interface in response to the collected sensor information. An activation unit is used to activate augmented reality display when the interaction state of the virtual object is detected to match the augmented reality range.
13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the augmented reality display method according to any one of claims 1 to 11.
14. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the augmented reality display method according to any one of claims 1 to 11.
15. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the augmented reality display method according to any one of claims 1 to 11.
Citation Information
Patent Citations
Information prompting method and device in virtual scene, electronic device and storage medium
CN112090069A
AR interaction state control method, device and equipment and storage medium
CN112148125A
KR20220052676A
