Display method and device for augmented reality, equipment and storage medium

Abstract

The invention discloses a display method and device for augmented reality, equipment and a storage medium, and belongs to the technical field of augmented reality. The method comprises the steps that in response to successful matching of first AR wearable equipment and second AR wearable equipment, a motion preparation interface is displayed, and the motion preparation interface is used for prompting whether target motion starts or not; in response to the received operation of determining to start the target motion, displaying the real-time motion scores of the first user and the second user in a scene environment corresponding to the first AR wearable device in an overlapping manner; displaying a virtual obstacle in a scene environment in an overlapping manner, wherein the virtual obstacle is determined by sensor data collected by a target sensor; and updating the real-time movement score of the first user based on the collision condition of the first user and the virtual obstacle. According to the scheme, the application mode of the AR wearable device in the motion scene is expanded, and the display effect and the interaction effect of the AR wearable device in the motion scene are improved.

Description

Technical Field

[0001] This application relates to the field of augmented reality technology, and particularly to a display method, device, equipment and storage medium for augmented reality. Background Art

[0002] An augmented reality (AR) wearable device is a device that can superimpose virtual elements on the basis of a real scene environment.

[0003] The matching degree between AR wearable devices and sports scenarios is relatively high. Correspondingly, there are also many applications in sports scenarios. In the related art, during the process of a user's movement, an AR wearable device superimposes and displays information such as the user's heart rate, exercise duration, calorie consumption, etc. on the basis of a real scene environment to facilitate the user to understand their own exercise status.

[0004] However, the display method and display content of the above AR wearable device in the sports scenario are relatively single, which affects the display effect of the AR wearable device. Summary of the Invention

[0005] This application provides a display method, device, equipment and storage medium for augmented reality. The technical solutions are as follows:

[0006] According to one aspect of this application, a display method for augmented reality is provided. The method is executed by a first augmented reality (AR) wearable device, and a target sensor is included in the first AR wearable device. The method includes:

[0007] In response to meeting the matching trigger condition, a matching interface is displayed, and the matching interface is used for the first AR wearable device to match other AR wearable devices;

[0008] In response to the successful matching of the first AR wearable device and the second AR wearable device, a sports preparation interface is displayed, and the sports preparation interface is used to prompt whether to start a target sport;

[0009] In response to receiving an operation to determine to start a target sport, real-time sports scores of the first user and the second user performing the target sport are superimposed and displayed on the scene environment corresponding to the first AR wearable device; the first user is the user corresponding to the first AR wearable device, and the second user is the user corresponding to the second AR wearable device;

[0010] In response to the first user starting to perform the target sport, virtual obstacles are superimposed and displayed in the scene environment corresponding to the first AR wearable device, and the virtual obstacles are determined by sensor data collected by the target sensor;

[0011] Update the real-time motion score of the first user based on the collision situation between the first user and the virtual obstacle.

[0012] According to one aspect of the present application, there is provided a display device for augmented reality, the device comprising:

[0013] A first interface display module, configured to display a matching interface in response to meeting a matching trigger condition, the matching interface being used for the first AR wearable device to match other AR wearable devices; the first AR wearable device includes a target sensor;

[0014] A second interface display module, configured to display a motion preparation interface in response to successful matching between the first AR wearable device and the second AR wearable device, the motion preparation interface being used to prompt whether to start a target motion;

[0015] A score display module, configured to superimpose and display the real-time motion scores of the first user and the second user performing the target motion on the scene environment corresponding to the first AR wearable device in response to receiving an operation to determine to start the target motion; the first user is the user corresponding to the first AR wearable device, and the second user is the user corresponding to the second AR wearable device;

[0016] An obstacle display module, configured to superimpose and display a virtual obstacle in the scene environment corresponding to the first AR wearable device in response to the first user starting to perform the target motion, the virtual obstacle being determined by sensor data collected by the target sensor;

[0017] An update module, configured to update the real-time motion score of the first user based on the collision situation between the first user and the virtual obstacle.

[0018] In some embodiments, the trigger condition includes: the first AR wearable device is located within the range of the place corresponding to the target motion;

[0019] The matching interface is used for the first AR wearable device to match other AR wearable devices within the range of the place.

[0020] In some embodiments, the first interface display module is configured to,

[0021] In response to the first AR wearable device being located within the range of the place, display a matching prompt message, the matching prompt message being used to prompt whether to send a matching request to other AR wearable devices within the range of the place;

[0022] In response to an operation of confirming to send a matching request to other AR wearable devices within the scope of the venue, display the matching interface;

[0023] The device further includes:

[0024] A first matching determination module, configured to determine that the matching with the second AR wearable device is successful in response to the second AR wearable device receiving an operation of determining to match with the first AR wearable device.

[0025] In some embodiments, the first interface display module is configured to display the matching interface in response to the first AR wearable device being within the scope of the venue and receiving a matching request sent by the second AR wearable device; the matching interface includes matching confirmation information, and the matching confirmation information is used to prompt whether to match with the second AR wearable device;

[0026] The device further includes:

[0027] A second matching determination module, configured to determine that the matching with the second AR wearable device is successful in response to receiving an operation of determining to match with the second AR wearable device.

[0028] In some embodiments, the obstacle display module is configured to,

[0029] Based on the sensor data collected by the target sensor, obtain display parameters;

[0030] Based on the display parameters, superimpose and display one or more of the virtual obstacles in the scene environment corresponding to the first AR wearable device.

[0031] In some embodiments, the obstacle display module is configured to,

[0032] Send the sensor data to the server;

[0033] Receive the display parameters returned by the server.

[0034] In some embodiments, a cloud game client corresponding to the target motion runs on the first AR wearable device; the obstacle display module is configured to,

[0035] Send the sensor data to the server corresponding to the cloud game client;

[0036] Through the cloud game client, render the one or more virtual obstacles based on the display parameters, and superimpose and display the rendered one or more virtual obstacles in the scene environment corresponding to the first AR wearable device.

[0037] In some embodiments, the display parameters include at least one of the following parameters:

[0038] The number of the virtual obstacles, and the size of the virtual obstacles.

[0039] In some embodiments, the target motion is a swimming motion, and the target sensor includes a pressure sensor;

[0040] In response to the display parameters including the number of the virtual obstacles, the number of the virtual obstacles is inversely correlated with the pressure value corresponding to the pressure sensor data collected by the pressure sensor;

[0041] In response to the display parameters including the size of the virtual obstacles, the size of the virtual obstacles is inversely correlated with the pressure value corresponding to the pressure sensor data collected by the pressure sensor.

[0042] In some embodiments, the target sensor includes a speed sensor;

[0043] In response to the display parameters including the number of the virtual obstacles, the number of the virtual obstacles is inversely correlated with the speed value corresponding to the speed sensor data collected by the speed sensor;

[0044] In response to the display parameters including the size of the virtual obstacles, the size of the virtual obstacles is inversely correlated with the speed value corresponding to the speed sensor data collected by the speed sensor.

[0045] In some embodiments, the target sensor includes a moving direction sensor;

[0046] In response to the display parameters including the number of the virtual obstacles, the number of the virtual obstacles is inversely correlated with the slope; the slope is determined by the angle between the moving direction corresponding to the moving direction sensor data collected by the moving direction sensor and the horizontal plane;

[0047] In response to the display parameters including the size of the virtual obstacles, the size of the virtual obstacles is inversely correlated with the slope.

[0048] According to another aspect of the present application, there is provided a computer device, which includes a processor and a memory. At least one program is stored in the memory, and the at least one program is loaded and executed by the processor to implement the display method for augmented reality as described in the above aspect.

[0049] According to another aspect of the present application, there is provided a computer-readable storage medium storing at least one program, and the at least one program is loaded and executed by a processor to implement the display method for augmented reality as described in the above aspect.

[0050] According to another aspect of the present application, there is provided a computer program product including computer instructions stored in a computer-readable storage medium, and a processor reads and executes the computer instructions from the computer-readable storage medium to implement the display method for augmented reality as described in the above aspect.

[0051] The beneficial effects brought by the technical solution provided by the present application at least include:

[0052] When the first AR wearable device meets the matching trigger condition, a matching interface for matching with other AR wearable devices is displayed, and when successfully matched with the second AR wearable device, a motion preparation interface for prompting whether to start a target motion is further displayed. After determining to start the target motion, real-time motion scores of each user corresponding to the AR wearable device during the target motion are superimposed and displayed on the scene environment, and virtual obstacles determined by sensor data collected by the target sensor are provided. The real-time motion scores are updated according to the collision situation between the user and the virtual obstacle. That is to say, the above solution reflects the interaction process of matching among multiple users and jointly performing the target motion through the display content of the AR wearable device, expands the application mode of the AR wearable device in the motion scene and the game mode of multiple users jointly performing the target motion based on the AR wearable device, and improves the display effect of the AR wearable device in the motion scene and the interaction effect among users during the target motion through the AR wearable device. Description of the Drawings

[0053] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0054] Figure 1 is a structural block diagram of a computer system provided by an exemplary embodiment of the present application;

[0055] Figure 2 is a flowchart of a display method for augmented reality provided by an exemplary embodiment of the present application;

[0056] Figure 3It is a flowchart of a display method for augmented reality provided by an exemplary embodiment of the present application;

[0057] Figure 4 It is a flowchart of a display method for augmented reality provided by an exemplary embodiment of the present application;

[0058] Figure 5 It is a flowchart of a display method for augmented reality provided by an exemplary embodiment of the present application;

[0059] Figure 6 It is a schematic diagram of the display of positioning information related to the present application;

[0060] Figure 7 It is a schematic diagram of the interface for actively initiating matching related to the present application;

[0061] Figure 8 It is a schematic diagram of the hardware related to the present application;

[0062] Figure 9 It is a schematic diagram of the interface for passive matching related to the present application;

[0063] Figure 10 It is a schematic diagram of the interface for starting the game related to the present application;

[0064] Figure 11 It is a schematic diagram of the display of virtual obstacles related to the present application;

[0065] Figure 12 It is a flowchart of the use of intelligent swimming goggles related to the present application;

[0066] Figure 13 It is a block diagram of a display device for augmented reality provided by an exemplary embodiment of the present application;

[0067] Figure 14 It is a block diagram of the structure of a computer device provided by an exemplary embodiment of the present application.

[0068] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Detailed Embodiments

[0069] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0070] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0071] The terms used in the present disclosure are for the purpose of describing particular embodiments only and are not intended to limit the present disclosure. The singular forms "a", "the", and "said" used in the present disclosure and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0072] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions. For example, the object behaviors such as attack operations involved in the present application are obtained under full authorization.

[0073] It should be understood that although the terms first, second, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first parameter may also be referred to as the second parameter, and similarly, the second parameter may also be referred to as the first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0074] Figure 1 The block diagram of the computer system provided by an exemplary embodiment of the present application is shown. The computer system 100 includes: a first terminal 110, a server 120; optionally, the computer system 100 may further include a second terminal 130.

[0075] The first terminal 110 installs and runs a client 111 that supports AR. When the first terminal runs the client 111, the user interface of the client 111 is displayed on the screen of the first terminal 110. The client 111 can be any one of a sports game, an Augmented Reality (AR) program, and an AR game. In this embodiment, the client 111 is taken as an example of a sports game. The first terminal 110 is the terminal used by the first user 112.

[0076] The second terminal 130 installs and runs a client 131 that supports AR. When the second terminal 130 runs the client 131, the user interface of the client 131 is displayed on the screen of the second terminal 130. The client can be any one of a sports game, an Augmented Reality (AR) program, and an AR game. In this embodiment, the client is taken as an example of a sports game. The second terminal 130 is the terminal used by the second user 132.

[0077] Optionally, the clients installed on the first terminal 110 and the second terminal 130 are the same, or the clients installed on the two terminals are of the same type on different operating system platforms (Android or IOS). The first terminal 110 can generally refer to one of multiple terminals, and the second terminal 130 can generally refer to another one of the multiple terminals. This embodiment only takes the first terminal 110 and the second terminal 130 as examples. The device types of the first terminal 110 and the second terminal 130 are the same or different, and the device types include at least one of: an AR wearable device, a terminal device (such as a smart phone) connected to the AR wearable device and providing computing support to the AR wearable device, a tablet computer, an e-book reader, an MP3 player, an MP4 player, a laptop computer, and a desktop computer.

[0078] As Figure 1 shown, taking the first terminal 110 as an AR wearable device as an example, the first terminal 110 includes: a head-mounted display 113, an augmented reality host 114, and a sensor 115.

[0079] The head-mounted display 113 is a display for wearing on the user's head for image display. The head-mounted display 113 generally includes a wearing part and a display part. The wearing part includes temple arms and an elastic band for wearing the head-mounted display 113 on the user's head. The display part includes a projection component and a lens. The projection component can project and display virtual elements on the lens, so as to simulate a reality augmentation effect of superimposing virtual elements in the real environment for the user.

[0080] Among them, the above-mentioned head-mounted display 113 and the augmented reality host 114 can be independent devices. For example, the head-mounted display 113 only has a display function, and the computing function is implemented by an independent augmented reality host 114 (such as an external mobile phone or computer). Alternatively, in other embodiments, the first terminal 110 can also be implemented as an AR all-in-one device, that is, the augmented reality host 114 that realizes the computing function for the user is integrated on the head-mounted display 113, and there is no need for an external mobile phone or PC host, and it can be used as soon as it is powered on. This embodiment will be described by taking the first terminal 110 implemented as an AR all-in-one device as an example.

[0081] Optionally, the sensor 115 may include a motion sensor for capturing the user's head movements so that the augmented reality host 114 changes the display screen in the head-mounted display 113.

[0082] The head-mounted display 113 is electrically connected to the augmented reality host 114 through a flexible circuit board, a hardware interface, a data cable, or a wireless network.

[0083] The augmented reality host 114 is used for modeling three-dimensional virtual objects, generating display elements corresponding to the three-dimensional virtual objects, etc. Of course, the augmented reality host 114 can also model two-dimensional virtual objects, generate display elements corresponding to the two-dimensional virtual objects, etc.

[0084] The augmented reality host 114 can receive the sensor data collected by the sensor 115 and generate virtual elements displayed on the head-mounted display 113 according to the sensor data. The augmented reality host 114 is usually implemented by electronic devices such as a processor, a memory, and an image augmented reality host arranged on a circuit board. Optionally, the augmented reality host 114 further includes an image acquisition device for capturing the actual scene picture in front of the user, and assisting in modeling three-dimensional virtual objects, generating display elements corresponding to the three-dimensional virtual objects, determining the display positions of the display elements, etc. according to the actual scene picture, so that the display elements superimposed and displayed on the actual scene environment can have a better fusion effect with the actual scene environment.

[0085] The augmented reality host 114 is connected to the sensor 115 through a cable, a Bluetooth connection, or a Wi-Fi (Wireless-Fidelity) connection.

[0086] The sensor 115 may include one or more of sensors such as a pressure sensor, a motion sensor, a moving direction sensor, a light sensor, a temperature sensor, and a gyroscope.

[0087] Figure 1Only two terminals are shown, but in different embodiments, there are multiple other terminals 140 that can access the server 120. Optionally, there is also one or more terminals 140 that are the terminals corresponding to the developers. A development and editing platform for the client that supports the augmented reality scenario is installed on the terminals 140. The developers can edit and update the client on the terminals 140, and transmit the updated client installation package to the server 120 through a wired or wireless network. The first terminal 110 and the second terminal 130 can download the client installation package from the server 120 to update the client.

[0088] The first terminal 110, the second terminal 130, and the other terminals 140 are connected to the server 120 through a wireless network or a wired network.

[0089] The server 120 includes at least one of a server, multiple servers, a cloud computing platform, and a virtualization center. The server 120 is used to provide background services for the client that supports the three-dimensional virtual scenario. Optionally, the server 120 undertakes the main computing work, and the terminal undertakes the secondary computing work; or, the server 120 undertakes the secondary computing work, and the terminal undertakes the main computing work; or, a distributed computing architecture is adopted between the server 120 and the terminal for collaborative computing.

[0090] In a schematic example, the server 120 includes a processor 122, a user account database 123, a scenario service module 124, and a user-oriented input / output interface (I / O interface) 125. Among them, the processor 122 is used to load the instructions stored in the server 120 and process the data in the user account database 123 and the scenario service module 124; the user account database 123 is used to store the data of the user accounts used by the first terminal 110, the second terminal 130, and the other terminals 140, such as the avatars of the user accounts, the nicknames of the user accounts, the combat power indexes of the user accounts, and the service areas where the user accounts are located; the scenario service module 124 is used to provide matches for users to play matching games or battles, such as 1V1 battles, level-breaking, etc.; the user-oriented I / O interface 125 is used to establish communication with the first terminal 110 and / or the second terminal 130 through a wireless network or a wired network to exchange data.

[0091] The solution for augmented reality provided in this application can be applied to, but is not limited to, at least one of the following sports scenarios: swimming, running, skiing, mountain climbing, skating, roller skating, and so on.

[0092] Figure 2 The flowchart of the display method for augmented reality provided by an exemplary embodiment of this application is shown. This method can be executed by a first AR wearable device, and the first AR wearable device can be Figure 1The first terminal 110 or the second terminal 130 in the illustrated implementation environment, and the first AR wearable device may include a target sensor. The method includes:

[0093] Step 210: In response to meeting the matching trigger condition, display a matching interface for the first AR wearable device to match with other AR wearable devices.

[0094] In an embodiment of the present application, when the above first AR wearable device is turned on and meets a preset trigger condition, it can be matched with other AR wearable devices so that the first user wearing the first AR wearable device and other users wearing AR wearable devices can exercise together.

[0095] Among them, the above display of the matching interface may refer to the first AR wearable device projecting and displaying the above matching interface on the lens of the first AR wearable device through a projection component. Through this solution, the first AR wearable device can directly display the matching interface in an AR manner without the need for an additional display device, improving the convenience for users to view the matching interface.

[0096] Alternatively, the above display of the matching interface may also refer to the first AR wearable device displaying the above matching interface on the display screen of other terminal devices (such as a smart phone) of the first user; for example, in response to meeting the matching trigger condition, the first AR wearable device transmits the information of the matching interface to the smart phone of the first user, and the smart phone of the first user displays the matching interface. Through this solution, the first AR wearable device can directly display the matching interface through an additional display device. Since the display content of the matching interface may include elements that require user interaction (such as confirmation controls), displaying the matching interface through other display devices (such as a smart phone) can be more convenient for users to operate, thereby improving the user interaction efficiency.

[0097] Step 220: In response to the successful matching of the first AR wearable device and the second AR wearable device, display a motion preparation interface for prompting whether to start the target motion.

[0098] In an embodiment of the present application, after the first AR wearable device and the second AR wearable device are successfully matched, in order to enable the users of the first AR wearable device and the second AR wearable device to start the target motion simultaneously, a motion preparation interface can be displayed to prompt the user to confirm whether to start the target motion, and at the same time provide the user with preparation time before the motion, thereby ensuring the interaction effect related to the target motion among users of different AR wearable devices.

[0099] For example, the above-mentioned exercise preparation interface may include text prompting the user whether to confirm starting the target exercise, as well as a confirmation control. The user can trigger the confirmation control to determine to start the target exercise. After the users of the first AR wearable device and the second AR wearable device respectively trigger the confirmation control, a countdown (such as a 5-second countdown) can be displayed. When the countdown ends, the user can start the target exercise, thereby ensuring the synchronization of the start of the target exercise among the users corresponding to the matching AR wearable devices.

[0100] Optionally, if the above-mentioned second AR wearable device includes multiple second AR wearable devices, in response to the successful matching of the first AR wearable device and the multiple second AR wearable devices, the exercise preparation interface is displayed. Correspondingly, the first AR wearable device can display a countdown when the users of the first AR wearable device and the multiple second AR wearable devices respectively confirm to start the target exercise.

[0101] Among them, the above-mentioned display of the exercise preparation interface may refer to the first AR wearable device projecting and displaying the above-mentioned exercise preparation interface on the lens of the first AR wearable device through a projection component. With this solution, no additional display device is required, improving the convenience for the user to view the exercise preparation interface.

[0102] Or, the above-mentioned display of the exercise preparation interface may also refer to the first AR wearable device displaying the above-mentioned exercise preparation interface on the display screen of other terminal devices (such as a smart phone) of the first user. For example, the first AR wearable device transmits the information of the exercise preparation interface to the smart phone of the first user, and the smart phone of the first user displays the exercise preparation interface. With this solution, it is more convenient for the user to operate, thereby improving the user's interaction efficiency.

[0103] Step 230: In response to receiving an operation to determine to start the target exercise, superimpose and display the real-time exercise scores of the first user and the second user performing the target exercise on the scene environment corresponding to the first AR wearable device; the first user is the user corresponding to the first AR wearable device, and the second user is the user corresponding to the second AR wearable device.

[0104] In the embodiment of the present application, the above-mentioned superimposing and displaying the real-time exercise scores of the first user and the second user performing the target exercise on the scene environment corresponding to the first AR wearable device may refer to the first AR wearable device projecting and displaying the real-time exercise scores of the first user and the second user performing the target exercise on the lens of the first AR wearable device through a projection component. Through the above processing, the AR wearable device can render a picture of multiple users performing target interactive exercises together, thereby providing the user with the fun of performing the target exercise and making the user not bored when performing the target exercise.

[0105] Optionally, if the above-mentioned second AR wearable device includes multiple second AR wearable devices, the first AR wearable device may superimpose and display the real-time motion scores of the first user and each of the multiple second users on the scene environment corresponding to the first AR wearable device.

[0106] Step 240: In response to the first user starting to perform a target motion, superimpose and display virtual obstacles in the scene environment corresponding to the first AR wearable device, where the virtual obstacles are determined by the sensor data collected by the target sensor.

[0107] In the embodiments of the present application, based on the user performing the target motion, the AR wearable device can also implement richer applications. Specifically, the AR wearable device can determine and display virtual obstacles according to the sensor data collected by the target sensor.

[0108] Step 250: Update the real-time motion score of the first user based on the collision situation between the first user and the virtual obstacle.

[0109] In a possible implementation manner of the embodiments of the present application, if the first user collides with a virtual obstacle, a specified real-time motion score can be deducted from the first user. For example, whenever the first user collides with a virtual obstacle, a deduction score corresponding to the virtual obstacle (such as 5 points) can be subtracted from the current real-time motion score of the first user. Among them, different virtual obstacles can correspond to the same deduction score or different deduction scores.

[0110] Optionally, in the case where different virtual obstacles correspond to different deduction scores, the deduction score corresponding to the virtual obstacle can be inversely correlated with the size of the virtual obstacle; that is, the larger the size of the virtual obstacle, the more difficult it is for the first user to avoid the virtual obstacle, and correspondingly, the lower the deduction score subtracted when the first user collides with the virtual obstacle. On the contrary, the smaller the size of the virtual obstacle, the easier it is for the first user to avoid the virtual obstacle, and correspondingly, the higher the deduction score subtracted when the first user collides with the virtual obstacle.

[0111] In another possible implementation manner of the embodiments of the present application, if the first user collides with a virtual obstacle, a specified real-time motion score can be added to the first user. For example, whenever the first user collides with a virtual obstacle, a first reward score corresponding to the virtual obstacle can be added to the current real-time motion score of the first user. Among them, different virtual obstacles can correspond to the same first reward score or different first reward scores.

[0112] Optionally, in the case where different virtual obstacles correspond to different first reward scores, the first reward score corresponding to a virtual obstacle may be inversely correlated with the size of the virtual obstacle; that is, the larger the size of the virtual obstacle, the lower the difficulty for the first user to collide with the virtual obstacle, and correspondingly, the lower the first reward score increased when the first user collides with the virtual obstacle. Conversely, the smaller the size of the virtual obstacle, the higher the difficulty for the first user to collide with the virtual obstacle, and correspondingly, the higher the first reward score increased when the first user collides with the virtual obstacle.

[0113] In another possible implementation manner of the embodiment of the present application, the virtual obstacles may form one or more virtual channels in the scene environment. If the first user passes through the virtual channel without colliding with the virtual obstacle, a specified real-time motion score may be added to the first user. For example, whenever the first user passes through a virtual channel without colliding with the virtual obstacle, the second reward score corresponding to the virtual channel may be added to the current real-time motion score of the first user. Among them, different virtual channels may correspond to the same second reward score or different second reward scores.

[0114] Optionally, in the case where different virtual channels correspond to different second reward scores, the second reward score corresponding to a virtual channel may be inversely correlated with the size of the virtual channel; that is, the larger the size of the virtual channel, the lower the difficulty for the first user to pass through the virtual channel without colliding with the virtual obstacle, and correspondingly, the lower the second reward score increased when the first user passes through the virtual channel without colliding with the virtual obstacle. Conversely, the smaller the size of the virtual channel, the higher the difficulty for the first user to pass through the virtual channel without colliding with the virtual obstacle, and correspondingly, the higher the second reward score increased when the first user passes through the virtual channel without colliding with the virtual obstacle.

[0115] Among them, the above several ways of updating the real-time motion score of the first user based on the collision situation between the first user and the virtual obstacle can be used alone or in combination. For example, the above virtual obstacles include at least one of the first type of virtual obstacle, the second type of virtual obstacle, and the third type of virtual obstacle. Among them, the first type of virtual obstacle is a virtual obstacle corresponding to a deduction score, the second type of virtual obstacle is a virtual obstacle corresponding to a reward score, and the third type of virtual obstacle is a virtual obstacle that forms a virtual channel.

[0116] In the embodiments of the present application, since the first user and the second user are matched through the AR wearable devices they wear respectively to perform the target movement, in the actual scenario environment, the first user and the second user are usually not face-to-face. In this case, in order to simulate the interaction effect among multiple users based on the target movement, in the embodiments of the present application, when the first user and the second user wearing the AR wearable devices perform the target movement, they can obtain real-time movement scores. Moreover, the AR wearable devices can, through the AR display method, display the real-time movement scores of the first user and the second user performing the target movement, so that the user can intuitively understand the movement progress or movement achievements of other users, thereby simulating the interaction effect among multiple users based on the target movement.

[0117] Meanwhile, in the solution shown in the embodiments of the present application, the AR wearable devices can determine and display virtual obstacles according to the sensor data collected by the target sensors, and update the real-time movement scores of the users according to the collision situations between the users and the virtual obstacles, thereby expanding the game mode in which users are matched through the AR wearable devices and perform the target movement together.

[0118] In a possible implementation manner, the above AR wearable devices can be used to run the client of the game corresponding to the above target movement, or can be used to display the screen of the game corresponding to the above target movement, and the above virtual obstacles are the virtual obstacles in the game corresponding to the target movement.

[0119] Optionally, the real-time movement scores of the above first user or second user can include two parts:

[0120] One part is the score of the first user or the second user performing the target movement, which can also be called the movement score, that is, the score calculated by the AR wearable device according to the movement situation of the current user. For example, the real-time movement score can be the score calculated by the AR wearable device according to data such as the movement speed, movement duration, and movement distance of the current user, and is used to represent the movement progress or movement achievements (movement ability) of the user;

[0121] The other part is the score determined based on the collision situation between the first user or the second user and the virtual obstacle, which can also be called the game score, that is, the reward score increased or the deduction score decreased according to the collision situation between the first user or the second user and the virtual obstacle.

[0122] Among them, when the first AR wearable device displays the real-time movement scores of the first user and the second user performing the target movement, it can display the movement score and the game score of each user among the first user and the second user respectively, so as to clearly display the scores of each user in the two scenarios of movement and game respectively, and improve the display effect of the scores.

[0123] When the first AR wearable device displays the real-time motion scores of the first user and the second user performing the target motion, it can also cumulatively display the motion scores and game scores of each of the first user and the second user, so as to display the comprehensive scores of each user in the two scenarios of motion and game, thereby enhancing the interest of interaction among multiple users based on the target motion.

[0124] Among them, the above AR wearable device can be an AR glasses. For example, the AR wearable device can be an AR glasses matching the target motion, such as AR swimming goggles corresponding to swimming, AR mountaineering glasses corresponding to mountaineering, AR skiing goggles corresponding to skiing, AR skiing windproof sunglasses corresponding to cycling, etc. Or, the above AR wearable device can also be a general AR glasses.

[0125] Among them, the AR wearable device can install the clients of games corresponding to various motions. Correspondingly, the AR wearable device can start the client of the game corresponding to the target motion according to the target motion.

[0126] For example, in the case where the above AR wearable device is a general AR glasses, the AR wearable device installs a swimming game client (corresponding to swimming), a mountaineering game client (corresponding to mountaineering), a skiing game client (corresponding to skiing), and a cycling game client (corresponding to cycling); in an exemplary solution, the above AR wearable device can, under the operation and control of the user, select to start the client of the game corresponding to the target motion. For example, the user can select to start one of the above game clients through operation methods such as touch buttons, gesture operations, and voice operations on the AR wearable device; in another exemplary embodiment, the AR wearable device can automatically start the corresponding game client according to the current location information. For example, the AR wearable device can obtain the current location information, determine the current sports venue according to the current location information, and start the game client corresponding to the sports venue. For example, when the AR wearable device detects that it is currently in a swimming pool, it can automatically start the swimming game client / prompt the user whether to confirm starting the swimming game client. When the AR wearable device detects that it is currently in a ski resort, it can automatically start the skiing game client / prompt the user whether to confirm starting the skiing game client.

[0127] In summary, in the solution shown in the embodiments of the present application, when the first AR wearable device meets the matching trigger condition, a matching interface for matching with other AR wearable devices is displayed, and in the case of successful matching with the second AR wearable device, a motion preparation interface for prompting whether to start a target motion is further displayed. After determining to start the target motion, real-time motion scores of each user corresponding to the AR wearable device are superimposed and displayed on the scene environment, and virtual obstacles determined by sensor data collected by the target sensor are generated. The real-time motion scores are updated according to the collision situation between the user and the virtual obstacles. That is to say, the above solution reflects the interaction process of matching among multiple users and performing a target motion together through the display content of the AR wearable device, expands the application mode of the AR wearable device in the motion scene and the game mode of multiple users performing a target motion together based on the AR wearable device, and improves the display effect of the AR wearable device in the motion scene and the interaction effect among users performing a target motion through the AR wearable device.

[0128] Based on the above Figure 2 shown embodiments, the above trigger condition may include that the user has performed a specified operation. For example, the user clicks a certain physical button on the AR wearable device, or inputs a predetermined trigger voice, or specifies a certain specified limb movement, etc. At this time, the matching range of the AR wearable device may not be limited to a specific area. For example, when the first AR wearable device meets the above trigger condition, it can be matched with other AR wearable devices at any location through a cloud server.

[0129] In the above embodiments, the user can perform the matching of the target motion through the AR wearable device without being restricted by distance, thereby expanding the way for multiple users to perform the target motion together and improving the interaction efficiency among users.

[0130] In other embodiments, based on the above Figure 2 shown solution, the above trigger condition may include: the first AR wearable device is within the range of the place corresponding to the target motion; the above matching interface is used for the first AR wearable device to match other AR wearable devices within the range of the place. At this time, the first AR wearable device can be matched with other AR wearable devices within the above range of the place through a cloud server, or the first AR wearable device can also be matched with other AR wearable devices within the above range of the place through short-range wireless communication technology (such as Bluetooth or Wi-Fi).

[0131] Among them, the above trigger condition can also be that the position of the AR wearable device is within a specified range. That is to say, when the AR wearable device is within the range of the venue corresponding to the target movement, it can be determined that the matching between AR wearable devices is triggered.

[0132] For example, an application corresponding to the target movement is installed in the AR wearable device. After the first user starts the application in the first AR wearable device, the application can obtain the position of the first AR wearable device. When the position of the first AR wearable device is within the range of the venue corresponding to the preset target movement, it can be determined that the trigger condition is met, and a matching interface can be displayed.

[0133] For another example, an application corresponding to the target movement is installed in the AR wearable device. When the position of the first AR wearable device is within the range of the venue corresponding to the preset target movement, it can be determined that the trigger condition is met, triggering the start of the above application and displaying a matching interface.

[0134] In the above embodiments of the present application, the matching between users can be automatically triggered by the range of the venue of the target movement, improving the matching method for multiple users to carry out the target movement together and improving the matching efficiency.

[0135] Figure 3 shows a flowchart of a display method for augmented reality provided by an exemplary embodiment of the present application. Based on the above Figure 2 In the shown solution, when the above trigger condition includes that the first AR wearable device is within the range of the venue corresponding to the target movement, and the matching interface is used to match other AR wearable devices within the venue range, as Figure 3 shown, the above step 210 may include step 210a and step 210b, and before the above step 220, step 212 is further included.

[0136] Step 210a: In response to the first AR wearable device being within the venue range, display a matching prompt message for prompting whether to send a matching request to other AR wearable devices within the venue range.

[0137] In an embodiment of the present application, when the first AR wearable device is within the range of the venue, it can prompt the first user whether to initiate a match actively. Specifically, the first AR wearable device can display a matching prompt message, which includes a prompt text on whether to send a matching request to other AR wearable devices within the venue range, as well as a confirmation control. The first user can trigger the confirmation control to send an operation to confirm sending a matching request to other AR wearable devices within the venue range. Optionally, the matching prompt message can also include a cancellation control, and the first user can trigger the cancellation control to send an operation to cancel sending a matching request to other AR wearable devices within the venue range.

[0138] Alternatively, in response to the first AR wearable device being within the range of the venue, when the first AR wearable device detects that there are other wearable devices within a specified range around it. For example, when obtaining the information of other wearable devices within the specified range around it pushed by the server, it is determined that there are other wearable devices within the specified range around it, or it detects whether there are other wearable devices within the specified range around it through short-range wireless communication. After that, the first AR wearable device can send a matching request to other wearable devices within the specified range around it.

[0139] Step 210b: In response to the operation of confirming to send a matching request to other AR wearable devices within the venue range, display a matching interface.

[0140] For example, after the first user triggers the confirmation control to send an operation to confirm sending a matching request to other AR wearable devices within the venue range, the first AR wearable device can send a matching request to other surrounding AR wearable devices through the server or in a short-range wireless communication manner.

[0141] Step 212: In response to the second AR wearable device receiving the operation of determining to match with the first AR wearable device, determine that the matching with the second AR wearable device is successful.

[0142] In an embodiment of the present application, after the second AR wearable device receives the matching request sent by the first AR wearable device, if the second user agrees to the match, it can send an operation to determine to match with the first AR wearable device. When the second AR wearable device receives the operation of determining to match with the first AR wearable device, it can feedback the information of receiving the operation of determining to match with the first AR wearable device to the first AR wearable device through the server or in a short-range wireless communication manner. After the first AR wearable device receives the feedback from the second AR wearable device, it can determine that the matching with the second AR wearable device is successful.

[0143] In the solution shown in the above embodiments of the present application, a solution is provided in which an AR wearable device actively initiates matching with other AR wearable devices, ensuring the controllability of the matching between AR wearable devices, and thus ensuring the interaction effect of the user's matching through the AR wearable device.

[0144] Figure 4 The flowchart of the display method for augmented reality provided by an exemplary embodiment of the present application is shown. Based on the above Figure 2 shown solution, in the case where the above trigger condition includes that the first AR wearable device is within the range of the place corresponding to the target movement, and the matching interface is used for the first AR wearable device to match other AR wearable devices within the range of the place, as Figure 4 shown, the above step 210 may include step 210c, and before the above step 220, step 214 is further included.

[0145] Step 210c: In response to the first AR wearable device being within the range of the place and receiving a matching request sent by the second AR wearable device, display the matching interface; the matching interface includes matching confirmation information, and the matching confirmation information is used to prompt whether to match with the second AR wearable device.

[0146] In the embodiment of the present application, when the first AR wearable device is within the range of the place, it can passively accept the matching, that is, receive the matching request sent by the second AR wearable device. At this time, the first AR wearable device can prompt the first user whether to accept the matching. Specifically, the first AR wearable device can display a matching interface including matching confirmation information, which includes a prompt text for whether to accept the matching request of other AR wearable devices within the range of the place, and a determination control. The first user can trigger the determination control to issue an operation to determine to match with the second AR wearable device; optionally, the matching prompt information can also include a cancellation control, and the first user can trigger the cancellation control to issue an operation to cancel the determination to match with the second AR wearable device.

[0147] Step 214: In response to receiving the operation to determine to match with the second AR wearable device, determine that the matching with the second AR wearable device is successful.

[0148] Among them, when the first user triggers the determination control to issue an operation to determine to match with the second AR wearable device, the first AR wearable device can immediately determine that the matching with the second AR wearable device is successful. At the same time, the first AR wearable device can also feedback the information of receiving the operation to determine to match with the second AR wearable device to the second AR wearable device, so that the AR wearable device can determine that the matching with the first AR wearable device is successful.

[0149] In the solution shown in the above embodiments of the present application, a solution is provided in which an AR wearable device initiatively initiates matching with other AR wearable devices passively, ensuring the controllability of the matching between AR wearable devices, and further ensuring the interaction effect of the user's matching through the AR wearable device.

[0150] Figure 5 The flowchart of the display method for augmented reality provided by an exemplary embodiment of the present application is shown. Based on the above Figures 2 to 4 For any of the solutions involved in the shown embodiments, step 240 may include step 240a and step 240b.

[0151] Step 240a: Obtain display parameters based on the sensor data collected by the target sensor.

[0152] In an exemplary solution, the AR wearable device can obtain the sensor data collected by the target sensor in real time and determine the display parameters locally according to the sensor data. This solution does not need to rely on the network, can ensure applicability even in the case of no network connection, and thus improves the applicable range of the solution.

[0153] In a possible implementation manner, obtaining display parameters based on the sensor data collected by the target sensor includes:

[0154] Send the sensor data to the server;

[0155] Receive the display parameters returned by the server.

[0156] In another exemplary solution, the AR wearable device can also upload the obtained sensor data to the server, and the server determines and returns the above display parameters according to the sensor data.

[0157] In the embodiments of the present application, the AR wearable device can report the sensor data to the server in the cloud. The server in the cloud determines the above display parameters according to the sensor data and feeds back the display parameters to the AR wearable device, reducing the computing power requirements of the AR wearable device and ensuring the computing efficiency of the display parameters of the virtual obstacle through cloud technology.

[0158] Step 240b: Superimpose and display one or more virtual obstacles in the scene environment corresponding to the first AR wearable device based on the display parameters.

[0159] In a possible implementation manner, the display parameters include at least one of the following parameters:

[0160] The number of virtual obstacles and the size of the virtual obstacles.

[0161] In an embodiment of the present application, the AR wearable device can obtain sensor data from a server or determine locally at least one of the number of virtual obstacles and the size of the virtual obstacles through local computing, and correspondingly display one or more virtual obstacles, thereby providing a solution to ensure that the display effect of the virtual obstacles is adapted to the sensor data collected by the AR wearable device.

[0162] Optionally, the position where the above one or more virtual obstacles are superimposed on the scene environment can be fixed or variable (that is, the virtual obstacles can move). Correspondingly, the above display parameters can also include parameters such as the display position and movement trajectory of the virtual obstacles.

[0163] Among them, based on the above display parameters, the AR wearable device / the game client running in the AR wearable device can display virtual obstacles adapted to the scene environment where the AR wearable device is currently located. For example, one or more image acquisition components can be provided in the AR wearable device. The AR wearable device can collect images of the scene environment around the AR wearable device through one or more image acquisition components, determine the area in the scene environment that meets the conditions for setting virtual obstacles based on the images of the scene environment, and combine at least one of the number and size of the virtual obstacles to determine the display position / movement trajectory of the virtual obstacles in the scene environment, and superimpose and display one or more virtual obstacles in the display scene environment according to the display position / movement trajectory.

[0164] For example, after the game client corresponding to the target movement in the AR wearable device obtains the image of the scene environment, it can identify the image through an image recognition model to determine the area in the scene environment that meets the conditions for setting virtual obstacles. Then, according to at least one of the number and size of the virtual obstacles, the virtual obstacles are evenly distributed to the area in the scene environment that meets the conditions for setting virtual obstacles to determine the display position / movement trajectory of the virtual obstacles in the scene environment. Among them, the above image recognition model can be obtained by supervised machine learning training in advance through scene environment image samples and labeled areas obtained by labeling the scene environment image samples.

[0165] For another example, when the above display parameters include not only the number and size of the virtual obstacles but also parameters such as the display position / movement trajectory of the virtual obstacles, the AR wearable device can superimpose and display the virtual obstacles in the scene environment according to the number, size, display position / movement trajectory of the virtual obstacles.

[0166] Among them, when the above AR wearable device superimposes and displays virtual obstacles in the scene environment according to the number, size, display position / movement trajectory of the virtual obstacles, it can detect the direction of the front facing of the AR wearable device, and determine whether there is a display position corresponding to the virtual obstacle within the area corresponding to the direction of the front facing of the AR wearable device (such as a conical area with a preset angle centered on the direction of the front facing) according to the number, size, display position / movement trajectory of the virtual obstacles. If there is a virtual obstacle within the area corresponding to the direction of the front facing, the position of the virtual obstacle in the scene environment is projected and displayed on the lens of the AR wearable device.

[0167] In a possible implementation manner, a cloud game client corresponding to the target motion runs in the first AR wearable device; the step of sending the sensor data to the server may include: sending the sensor data to the server corresponding to the cloud game client;

[0168] The step of superimposing and displaying one or more virtual obstacles in the scene environment corresponding to the first AR wearable device based on the display parameters may include:

[0169] Render one or more virtual obstacles based on the display parameters through the cloud game client, and superimpose and display the rendered one or more virtual obstacles in the scene environment corresponding to the first AR wearable device.

[0170] In the embodiment of the present application, during the operation of the cloud game client in the above AR wearable device, the target sensor in the AR wearable device can collect sensor data and transfer the sensor data to the cloud game client. The cloud game client uploads the sensor data to the server corresponding to the cloud game client (which can be called the cloud game server). After receiving the sensor data uploaded by the cloud game client, the cloud game server can determine the corresponding display parameters according to the sensor data and in combination with the pre-set parameter setting rules, and send the display parameters to the cloud game client in the AR wearable device. The cloud game client renders and displays the virtual obstacles according to the display parameters.

[0171] Among them, the above parameter setting rules may include the correspondence between the sensor data and the display parameters. For example, the above parameter setting rules may be a correspondence table between the data values of the sensor data and the parameter values of the display parameters, or the above parameter setting rules may also be a calculation formula / machine learning model for calculating the display parameters by inputting the sensor data, etc.

[0172] Among them, the above-mentioned server can also determine the above-mentioned display parameters in combination with the image of the scene environment where the AR wearable device is located. For example, after the client of the game corresponding to the target movement in the AR wearable device obtains the image of the scene environment, it can also upload the image to the server. The server combines the above-mentioned sensor data to determine the above-mentioned display parameters. At this time, the display parameters can include one or more of the number of virtual obstacles, the size of the virtual obstacles, the display position or the movement trajectory of the virtual obstacles. For example, when the AR wearable device sends the sensor data to the server, it also uploads the image of the scene environment to the server.

[0173] In an exemplary solution, the above-mentioned server can first determine at least one of the number of virtual obstacles and the size of the virtual obstacles in the above-mentioned display parameters through the above-mentioned sensor data. Then, according to the image of the scene environment, determine the area in the scene environment that meets the conditions for setting virtual obstacles, and combine at least one of the number and size of the virtual obstacles to determine the display position / movement trajectory of the virtual obstacles in the scene environment. This process can refer to the process of the above-mentioned AR wearable device determining the display position / movement trajectory of the virtual obstacles in the scene environment locally, which will not be elaborated here.

[0174] In another exemplary solution, the server can also input the above-mentioned sensor data and the image of the scene environment into a display parameter prediction model to obtain the display parameters output by the display parameter prediction model. Among them, the display parameter prediction model can be obtained by supervised machine learning training in advance through sensor data samples, scene environment image samples, and labeled display parameters obtained by pre-labeling the sensor data samples and scene environment image samples.

[0175] In addition, since the first user and the second user are matched through their respective AR wearable devices to perform the target movement, the movement environments faced by the first user and the second user may not be the same (for example, the spatial sizes of the target movements of the first user and the second user are different, the density of the surrounding people is different, etc.), or the ways in which the first user and the second user perform the target movement may be different (for example, the postures or routes of the target movements of the first user and the second user are different). Or rather, it may be difficult for the first user and the second user to perform a fair target movement through a unified movement environment or movement method. At this time, the sensor data collected by the target sensors in the respective AR wearable devices of the first user and the second user can display different virtual obstacles in the respective AR wearable devices of the first user and the second user, so as to balance the difficulty of the first user and the second user performing the target movement respectively, thereby further improving the interaction effect when the first user and the second user perform the target movement.

[0176] Specifically, during the process of the target movement, the AR wearable device can determine the display parameters of the virtual obstacle in real time according to the sensor data currently collected by the target sensor, and display the virtual obstacle according to the display parameters, so as to improve the diversity of the virtual obstacle and further improve the display effect of the virtual obstacle.

[0177] For example, the above sensor data can be one or more of a pressure sensor representing water pressure / atmospheric pressure, speed sensor data representing the moving speed, and a moving direction sensor data representing the moving direction.

[0178] For example, in some embodiments, the target movement is a swimming movement, and the target sensor includes a pressure sensor;

[0179] In response to the display parameter including the number of virtual obstacles, the number of virtual obstacles is inversely correlated with the pressure value corresponding to the pressure sensor data collected by the pressure sensor;

[0180] In response to the display parameter including the size of the virtual obstacle, the size of the virtual obstacle is inversely correlated with the pressure value corresponding to the pressure sensor data collected by the pressure sensor.

[0181] In the embodiments of the present application, during the swimming movement, different users may use different swimming postures. For example, some users like diving, and some users like breaststroke. However, the difficulty of diving and breaststroke is different. Simply put, the deeper the user dives, the greater the difficulty of swimming. In this regard, in the embodiments of the present application, when the target movement is a swimming movement, in order to balance the swimming difficulty of different users, the display parameters of the virtual obstacle can be determined through the pressure sensor data collected by the pressure sensor on the AR wearable device. Specifically, the greater the pressure value corresponding to the pressure sensor data, the deeper the depth. At this time, virtual obstacles with lower difficulty need to be provided to the user, that is, virtual obstacles with fewer numbers and smaller sizes are set; on the contrary, the smaller the pressure value corresponding to the pressure sensor data, the shallower the depth. At this time, virtual obstacles with higher difficulty need to be provided to the user, that is, virtual obstacles with more numbers and larger sizes are set, so as to improve the fairness of the swimming competition between users and improve the interaction effect of the swimming movement between users.

[0182] For another example, in some embodiments, the target sensor includes a speed sensor;

[0183] In response to the display parameter including the number of virtual obstacles, the number of virtual obstacles is inversely correlated with the speed value corresponding to the speed sensor data collected by the speed sensor;

[0184] In response to the display parameter including the size of the virtual obstacle, the size of the virtual obstacle is inversely correlated with the speed value corresponding to the speed sensor data collected by the speed sensor.

[0185] In the embodiments of the present application, during exercises such as running and skiing, different users may exercise at different speeds. For example, some users like to run fast, while some users like to run slowly. However, the difficulty levels of running fast and running slowly are different. Simply put, the deeper a user runs, the greater the difficulty of running. In this regard, in the embodiments of the present application, when the target exercise is running, skiing, etc., in order to balance the running difficulty of different users, the display parameters of the virtual obstacle can be determined based on the speed sensor data collected by the speed sensor on the AR wearable device. Specifically, the larger the speed value corresponding to the speed sensor data, the faster the user's speed. At this time, virtual obstacles with a lower difficulty level need to be provided to the user, that is, virtual obstacles with a smaller number and smaller size are set; on the contrary, the smaller the speed value corresponding to the speed sensor data, the slower the user's speed. At this time, virtual obstacles with a higher difficulty level need to be provided to the user, that is, virtual obstacles with a larger number and larger size are set, thereby improving the fairness of running / skiing and other sports competitions among users and enhancing the interaction effect among users.

[0186] For another example, in some embodiments, the target sensor includes a movement direction sensor;

[0187] In response to the display parameter including the number of virtual obstacles, the number of virtual obstacles is inversely correlated with the slope; the slope is determined by the angle between the movement direction corresponding to the movement direction sensor data collected by the movement direction sensor and the horizontal plane;

[0188] In response to the display parameter including the size of the virtual obstacle, the size of the virtual obstacle is inversely correlated with the slope.

[0189] In the embodiments of the present application, in sports such as mountaineering and skiing, different users may use different slopes for sports. For example, some users like to ski at positions with a larger slope, while some users like to ski at positions with a smaller slope. However, the difficulty of skiing is different for different slopes. Simply put, the larger the slope, the greater the difficulty of skiing. In this regard, in the embodiments of the present application, when the target sport is mountaineering, skiing, etc., in order to balance the sports difficulty of different users, the display parameters of virtual obstacles can be determined based on the slope corresponding to the mobile direction sensor data collected by the mobile direction sensor on the AR wearable device. Specifically, the larger the slope corresponding to the mobile direction sensor data, the greater the skiing difficulty of the user. At this time, virtual obstacles with lower difficulty need to be provided to the user, that is, virtual obstacles with a smaller number and smaller size are set; on the contrary, the smaller the slope corresponding to the mobile direction sensor data, the smaller the skiing difficulty of the user. At this time, virtual obstacles with higher difficulty need to be provided to the user, that is, virtual obstacles with a larger number and larger size are set, so as to improve the fairness of mountaineering / skiing and other sports competitions among users and enhance the interaction effect among users.

[0190] The solution shown in the above embodiments of the present application can be applied to AR wearable devices in various sports scenarios to expand the application methods of AR wearable devices in sports scenarios.

[0191] For example, taking the AR wearable device as an AR goggle (or called a smart AR goggle, smart goggle or smart AR goggle device, etc.), based on the solution shown in the above embodiments of the present application, by installing the client of cloud games on the goggle hardware device, the motion state of the user is detected in real time, and through the cooperation of the client and the server, the real-time virtual scene of the goggle is updated, enabling the user to swim and exercise while playing virtual reality game competitions through the smart AR goggle device, enhancing the fun and experience of the game, and increasing the social stickiness of the user. For example, the product corresponding to the AR goggle can be as follows:

[0192] 1) Swimming equipment: 1 smart goggle;

[0193] 2) Game players: 4;

[0194] 3) Input device: Smart goggle;

[0195] 4) Game: Multiplayer AR swimming competition game on the cloud game smart goggle;

[0196] 5) Control method: Wear the goggles and, in the AR game, avoid obstacles according to the change in water pressure detected by the water pressure sensor to see who reaches the finish line first;

[0197] 6) Extended gameplay: In addition to the fastest reaching the finish line mentioned above, it can also be the length of the swimming distance, or the success rate of avoiding obstacles at the same distance, etc.

[0198] From the product side, the usage method of the above AR goggles can be as follows:

[0199] S1, The user comes to xx swimming pool and puts on the smart goggles. At this time, the smart goggles locate xx swimming pool based on geographical location and display the location information. Optionally, this behavior information can refer to Figure 6 the shown location information display schematic diagram. Optionally, the Figure 6 location information 610 in it can disappear automatically in 3 seconds.

[0200] S2, Refer to Figure 7 , which shows the schematic diagram of the interface for actively initiating a match involved in this application. As shown in part (a) of Figure 7 , after the user puts on the smart goggles, a dialog box pops up on the goggle lens, showing: "Search for swimming partners and start a swimming race?" After pressing the confirmation button on the side of the hardware, start searching and enter the search interface. Or, as shown in part (b) of Figure 7 , when two swimmers are approaching, a message will pop up on the goggle lens indicating that other swimmers are approaching. Send a race invitation through the button on the goggles, and the invited person can see the invitation pop-up window on their glasses.

[0201] Refer to Figure 8 , which shows the hardware schematic diagram involved in this application. As shown in Figure 8 , through the button 810 on the smart goggles hardware, a race invitation can be sent to others; Refer to Figure 9 , which shows the schematic diagram of the interface for passive matching involved in this application. As shown in Figure 9 , the invited person can see the invitation pop-up window on their glasses. When the user selects to confirm and join immediately, the matching can be completed.

[0202] S3, Refer to Figure 10 , which shows the schematic diagram of the interface for starting the game involved in this application. As shown in Figure 10 , after the number of people is in place, for example, reaching the upper limit of 4 people and all 4 people are ready, each user clicks the start game button to enter the race.

[0203] S4, After that, all users swim in the real physical world while being able to see various parameters during the race on the lens, such as real-time ranking, speed, heart rate, rewards, etc. The goggles are equipped with a water pressure sensor, and the game scene will change to a certain extent according to the change of water pressure. The goggles will send the water pressure data to the cloud server, and the cloud server will render different game pictures according to different water pressure data. For example, please refer toFigure 11 , which shows a schematic diagram of the display of the virtual obstacle involved in the present application. As Figure 11 shown, if the player is swimming underwater, the water pressure is high and the swimming difficulty is high, and there are fewer virtual obstacles 1101 in the rendered game scene. The player can swim in a straight line. If the player is swimming on the water surface, there will be many virtual obstacles 1102 in the virtual scene. Hitting them will result in point deductions, and the player needs to swim around them.

[0204] Please refer to Figure 12 , which shows a flowchart of the use of the intelligent swimming goggles involved in the present application. As Figure 12 shown, from a technical perspective, the use process of the intelligent swimming goggles can be as follows:

[0205] The technical process is as follows:

[0206] S1, User 1 starts the swimming goggles client. The client detects the current real-time geographical location and displays it on the interface.

[0207] S2, At the same time, the client detects the user's state through the sensor. If User 1 wears it, the client prompts User 1 to turn on the Bluetooth and search for other User 2.

[0208] S3, Other User 2 needs to manually turn on the Bluetooth. After turning it on, User 1's swimming goggles client can search for it through the Bluetooth. At this time, User 1's client will send an instruction to invite the game through the Bluetooth. After Other User 2 receives it, the swimming goggles will display an invitation pop-up window on the interface, and User 2 agrees by clicking the side button.

[0209] S4, After User 2 agrees, it also sends the instruction back to User 1's swimming goggles client through the Bluetooth. When the number of users reaches the upper limit, User 1's swimming goggles client notifies other users and displays the start game interface. After Other User 2 receives it, it displays the start game interface.

[0210] S5, User 1's swimming goggles client will also display a scoring system selection. There are multiple scoring systems, such as distance system, speed system, obstacle success rate, etc. After User 1 selects, all users need to click the "Start Game" button. After clicking, all swimming goggles clients will send the instruction to the server. At the same time, User 1's swimming goggles client will also synchronize the information of all teammates to the server, and the server marks the information of all users in the current game.

[0211] S6, The user starts swimming, and all sensors of the swimming goggles are in the real-time on state for detecting. When the water pressure changes, it notifies the server. The server sends the specified scene parameters to the client according to the current pressure magnitude. After the client receives it, it displays different obstacles according to different scenes. When the scene changes, it also notifies the server. The server will record the time of the user in different scenes and calculate the user score.

[0212] S7, The swimming goggles detect the user's real-time route in real time. If a virtual obstacle is hit, the server will be notified to deduct points. The server will judge the difficulty level of the user's current scenario and determine the proportion of points to be deducted;

[0213] S8, At the same time, the swimming goggles also need to read the user's geographical location in real time, calculate the user's real-time swimming speed, and send the speed to the server. The server records the user's speed in real time and updates the score;

[0214] S9, If a user ends the game, the server will count the score of the current user. The scoring rules are based on the mode selected by the user 1, such as speed system, obstacle avoidance success rate, distance system, etc. Finally, when all users have ended the game, the server will obtain the scores of all users;

[0215] S10, The server sends the scores and rankings of all users to the swimming goggles client for display.

[0216] The solution shown in the above embodiments of the present application combines the intelligent swimming goggles with the game scenario, simulates the virtual scenario into the real world, and the user can experience the game scenario in the real world through the swimming goggles, improving the user experience and interest of the game.

[0217] Figure 13 The block diagram of a display device for augmented reality provided by an exemplary embodiment of the present application is shown. The device includes:

[0218] The first interface display module 1301 is configured to display a matching interface in response to meeting a matching trigger condition. The matching interface is used for the first AR wearable device to match other AR wearable devices; the first AR wearable device includes a target sensor;

[0219] The second interface display module 1302 is configured to display a motion preparation interface in response to the successful matching of the first AR wearable device and the second AR wearable device. The motion preparation interface is used to prompt whether to start the target motion;

[0220] The score display module 1303 is configured to, in response to receiving an operation to determine to start the target motion, superimpose and display the real-time motion scores of the first user and the second user performing the target motion on the scene environment corresponding to the first AR wearable device; the first user is the user corresponding to the first AR wearable device, and the second user is the user corresponding to the second AR wearable device;

[0221] An obstacle display module 1304, configured to, in response to the first user starting to perform the target movement, superimpose and display virtual obstacles in the scene environment corresponding to the first AR wearable device, where the virtual obstacles are determined based on the sensor data collected by the target sensor;

[0222] An update module 1305, configured to update the real-time movement score of the first user based on the collision situation between the first user and the virtual obstacle.

[0223] In some embodiments, the triggering condition includes: the first AR wearable device is within the range of the venue corresponding to the target movement;

[0224] The matching interface is used for the first AR wearable device to match other AR wearable devices within the range of the venue.

[0225] In some embodiments, the first interface display module 1301 is configured to,

[0226] In response to the first AR wearable device being within the range of the venue, display a matching prompt message, where the matching prompt message is used to prompt whether to send a matching request to other AR wearable devices within the range of the venue;

[0227] In response to an operation of confirming to send a matching request to other AR wearable devices within the range of the venue, display the matching interface;

[0228] The device further includes:

[0229] A first matching determination module, configured to, in response to the second AR wearable device receiving an operation of determining to match with the first AR wearable device, determine that the matching with the second AR wearable device is successful.

[0230] In some embodiments, the first interface display module 1301 is configured to, in response to the first AR wearable device being within the range of the venue and receiving a matching request sent by the second AR wearable device, display the matching interface; the matching interface includes matching confirmation information, where the matching confirmation information is used to prompt whether to match with the second AR wearable device;

[0231] The device further includes:

[0232] A second matching determination module, configured to, in response to receiving an operation of determining to match with the second AR wearable device, determine that the matching with the second AR wearable device is successful.

[0233] In some embodiments, the obstacle display module 1304 is configured to,

[0234] Obtain display parameters based on the sensor data collected by the target sensor;

[0235] Based on the display parameters, superimpose and display one or more of the virtual obstacles in the scene environment corresponding to the first AR wearable device.

[0236] In some embodiments, the obstacle display module 1304 is configured to,

[0237] Send the sensor data to the server;

[0238] Receive the display parameters returned by the server.

[0239] In some embodiments, a cloud game client corresponding to the target motion runs in the first AR wearable device; the obstacle display module 1304 is configured to,

[0240] Send the sensor data to the server corresponding to the cloud game client;

[0241] Based on the display parameters, render the one or more virtual obstacles through the cloud game client, and superimpose and display the rendered one or more virtual obstacles in the scene environment corresponding to the first AR wearable device.

[0242] In some embodiments, the display parameters include at least one of the following parameters:

[0243] The number of the virtual obstacles, and the size of the virtual obstacles.

[0244] In some embodiments, the target motion is a swimming motion, and the target sensor includes a pressure sensor;

[0245] In response to the display parameters including the number of the virtual obstacles, the number of the virtual obstacles is inversely correlated with the pressure value corresponding to the pressure sensor data collected by the pressure sensor;

[0246] In response to the display parameters including the size of the virtual obstacles, the size of the virtual obstacles is inversely correlated with the pressure value corresponding to the pressure sensor data collected by the pressure sensor.

[0247] In some embodiments, the target sensor includes a speed sensor;

[0248] In response to the display parameters including the number of the virtual obstacles, the number of the virtual obstacles is inversely correlated with the speed value corresponding to the speed sensor data collected by the speed sensor;

[0249] In response to the display parameter including the size of the virtual obstacle, the size of the virtual obstacle is inversely correlated with the speed value corresponding to the speed sensor data collected by the speed sensor.

[0250] In some embodiments, the target sensor includes a moving direction sensor;

[0251] In response to the display parameter including the number of virtual obstacles, the number of virtual obstacles is inversely correlated with the slope; the slope is determined by the angle between the moving direction corresponding to the moving direction sensor data collected by the moving direction sensor and the horizontal plane;

[0252] In response to the display parameter including the size of the virtual obstacle, the size of the virtual obstacle is inversely correlated with the slope.

[0253] It should be noted that when the device provided in the above embodiments realizes its functions, only the division of the above-mentioned respective functional modules is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to actual needs, that is, the content structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0254] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method; the technical effects obtained by each module performing operations are the same as those in the embodiments related to the method, and will not be elaborated here in detail.

[0255] Figure 14 The block diagram of a computer device 1400 provided by an exemplary embodiment of the present application is shown. The computer device 1400 may be an AR wearable device, such as: AR goggles, AR mountaineering glasses, AR ski goggles, AR ski windproof sunglasses, general AR glasses, and the like.

[0256] Generally, the computer device 1400 includes: a processor 1401 and a memory 1402.

[0257] The memory 1402 may include one or more computer-readable storage media, and the computer-readable storage media may be tangible and non-transitory. The memory 1402 may further include high-speed random access memory, as well as non-volatile memory, such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1402 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 1401 to implement the display method for augmented reality provided in the embodiments of the present application.

[0258] In some embodiments, the computer device 1400 may further optionally include: a peripheral device interface 1403 and at least one peripheral device. Specifically, the peripheral device includes at least one of: a radio frequency circuit 1404, a touch display screen 1405, a camera 1406, an audio circuit 1407, and a power supply 1408.

[0259] In some embodiments, the computer device 1400 further includes one or more sensors 1409. The one or more sensors 1409 include, but are not limited to: an acceleration sensor 1410, a gyroscope sensor 1411, a pressure sensor 1412, an optical sensor 1413, and a proximity sensor 1414.

[0260] Those skilled in the art can understand that the structures shown above do not constitute a limitation on the computer device 1400, and it may include more or fewer components than shown in the figure, or combine certain components, or adopt different component arrangements.

[0261] In an exemplary embodiment, a chip is further provided. The chip includes programmable logic circuits and / or program instructions, which are used to implement the display method for augmented reality described in the above aspects when the chip runs on a computer device.

[0262] In an exemplary embodiment, a computer program product is further provided. The computer program product includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor reads and executes the computer instructions to implement the display method for augmented reality provided by the above method embodiments.

[0263] In an exemplary embodiment, a computer-readable storage medium is further provided. The computer-readable storage medium stores a computer program, and the computer program is loaded and executed by a processor to implement the display method for augmented reality provided by the above method embodiments.

[0264] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a disk, or an optical disc, etc.

[0265] Those skilled in the art should be able to realize that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented by hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. The computer-readable medium includes computer storage media and communication media, where the communication media includes any medium that facilitates the transmission of a computer program from one place to another. The storage media can be any available medium accessible by a general-purpose or special-purpose computer.

[0266] The above are only alternative embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A display method for augmented reality, characterized in that, the method is executed by a first augmented reality (AR) wearable device, and the first AR wearable device includes a target sensor. The method includes: responding to meeting a matching trigger condition, and displaying a matching interface for the first AR wearable device to match other AR wearable devices; responding to successful matching between the first AR wearable device and a second AR wearable device, and displaying a motion preparation interface for prompting whether to start a target motion; responding to receiving an operation to determine to start the target motion, and superimposing and displaying real-time motion scores of a first user and a second user performing the target motion on the scene environment corresponding to the first AR wearable device; the first user is the user corresponding to the first AR wearable device, and the second user is the user corresponding to the second AR wearable device; responding to the first user starting to perform the target motion, and superimposing and displaying virtual obstacles in the scene environment corresponding to the first AR wearable device, where the virtual obstacles are determined by sensor data collected by the target sensor; updating the real-time motion score of the first user based on the collision situation between the first user and the virtual obstacles.

2. The method according to claim 1, characterized in that, the trigger condition includes: the first AR wearable device is located within the range of the place corresponding to the target motion; the matching interface is for the first AR wearable device to match other AR wearable devices within the range of the place.

3. The method according to claim 2, characterized in that, the responding to meeting the matching trigger condition and displaying the matching interface includes: responding to the first AR wearable device being located within the range of the place, and displaying a matching prompt message for prompting whether to send a matching request to other AR wearable devices within the range of the place; responding to an operation to confirm sending a matching request to other AR wearable devices within the range of the place, and displaying the matching interface; the method further includes: responding to the second AR wearable device receiving an operation to determine to match with the first AR wearable device, and determining successful matching with the second AR wearable device.

4. The method according to claim 2, characterized in that, the responding to meeting the matching trigger condition and displaying the matching interface includes: responding to the first AR wearable device being located within the range of the place and receiving a matching request sent by the second AR wearable device, and displaying the matching interface; the matching interface includes matching confirmation information for prompting whether to match with the second AR wearable device; the method further includes: responding to receiving an operation to determine to match with the second AR wearable device, and determining successful matching with the second AR wearable device.

5. The method according to any one of claims 1 to 4, characterized in that, Overlaying and displaying virtual obstacles in the scene environment corresponding to the first AR wearable device includes: Obtaining display parameters based on the sensor data collected by the target sensor; Based on the display parameters, overlaying and displaying one or more of the virtual obstacles in the scene environment corresponding to the first AR wearable device.

6. The method according to claim 5, wherein, The obtaining display parameters based on the sensor data collected by the target sensor includes: Sending the sensor data to the server; Receiving the display parameters returned by the server.

7. The method according to claim 6, wherein, A cloud game client corresponding to the target motion is running in the first AR wearable device; The sending the sensor data to the server includes: Sending the sensor data to the server corresponding to the cloud game client; The overlaying and displaying one or more of the virtual obstacles in the scene environment corresponding to the first AR wearable device based on the display parameters includes: Through the cloud game client, rendering the one or more virtual obstacles based on the display parameters, and overlaying and displaying the rendered one or more virtual obstacles in the scene environment corresponding to the first AR wearable device.

8. The method according to claim 6, wherein, The display parameters include at least one of the following parameters: The number of the virtual obstacles and the size of the virtual obstacles.

9. The method according to claim 8, wherein, The target motion is a swimming motion, and the target sensor includes a pressure sensor; In response to the display parameters including the number of the virtual obstacles, the number of the virtual obstacles is inversely correlated with the pressure value corresponding to the pressure sensor data collected by the pressure sensor; In response to the display parameters including the size of the virtual obstacles, the size of the virtual obstacles is inversely correlated with the pressure value corresponding to the pressure sensor data collected by the pressure sensor.

10. The method according to claim 8, wherein, The target sensor includes a speed sensor; In response to the display parameters including the number of the virtual obstacles, the number of the virtual obstacles is inversely correlated with the speed value corresponding to the speed sensor data collected by the speed sensor; In response to the display parameters including the size of the virtual obstacles, the size of the virtual obstacles is inversely correlated with the speed value corresponding to the speed sensor data collected by the speed sensor.

11. The method according to claim 8, wherein, The target sensor includes a moving direction sensor; In response to the display parameters including the number of the virtual obstacles, the number of the virtual obstacles is inversely correlated with the slope; the slope is determined by the angle between the moving direction corresponding to the moving direction sensor data collected by the moving direction sensor and the horizontal plane; In response to the display parameters including the size of the virtual obstacles, the size of the virtual obstacles is inversely correlated with the slope.

12. A display device for augmented reality, characterized in that, the device comprises: A first interface display module, configured to display a matching interface in response to meeting a matching trigger condition, where the matching interface is for a first AR wearable device to match other AR wearable devices; a target sensor is included in the first AR wearable device; A second interface display module, configured to display a motion preparation interface in response to successful matching between the first AR wearable device and a second AR wearable device, where the motion preparation interface is for prompting whether to start a target motion; A score display module, configured to, in response to receiving an operation to determine to start the target motion, superimpose and display real-time motion scores of a first user and a second user performing the target motion in the scene environment corresponding to the first AR wearable device; the first user is the user corresponding to the first AR wearable device, and the second user is the user corresponding to the second AR wearable device; An obstacle display module, configured to, in response to the first user starting to perform the target motion, superimpose and display virtual obstacles in the scene environment corresponding to the first AR wearable device, where the virtual obstacles are determined by sensor data collected by the target sensor; An update module, configured to update the real-time motion score of the first user based on the collision situation between the first user and the virtual obstacles.

13. A computer device, characterized in that, the computer device comprises: a processor and a memory, where at least one segment of program is stored in the memory; the processor is configured to execute the at least one segment of program in the memory to implement the display method for augmented reality according to any one of claims 1 to 11 above.

14. A computer-readable storage medium, characterized in that, executable instructions are stored in the readable storage medium, and the executable instructions are loaded and executed by a processor to implement the display method for augmented reality according to any one of claims 1 to 11 above.

15. A computer program product, characterized in that, the computer program product comprises computer instructions, the computer instructions are stored in a computer-readable storage medium, and a processor reads and executes the computer instructions from the computer-readable storage medium to implement the display method for augmented reality according to any one of claims 1 to 11 above.

Images