Display control method applied to mixed reality system and mixed reality system

Abstract

The embodiment of the invention provides a display control method applied to a mixed reality system and the mixed reality system. The multiple pieces of mixed reality glasses construct a unified target coordinate system based on the three mark points; the mixed reality glasses worn by the player identify the pose of the entity card played by the player in the uniformly constructed target coordinate system, the pose is sent to the mixed reality glasses worn by other players, and the mixed reality glasses of other players convert the pose into the pose of the virtual scene under the world coordinate system displayed locally. And displaying a virtual animation corresponding to the card in a virtual scene according to the converted pose. According to the technical scheme provided by the embodiment of the invention, immersive offline card board game experience can be brought to players.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular, to a display control method and a mixed reality system applied to a mixed reality system. Background Art

[0002] Card games, also known as playing card games, belong to a type of tabletop game. This game requires the participation of multiple players.

[0003] However, traditional offline card games do not have any immersive experience. Summary of the Invention

[0004] In view of the above problems, this application is proposed to provide a display control method and a mixed reality system applied to a mixed reality system that solve the above problems or at least partially solve the above problems.

[0005] In the first aspect of this application, a display control method applied to a mixed reality system is provided. The mixed reality system includes: a first mixed reality glasses, a second mixed reality glasses, and three marker points set in a real scene, and the three marker points are not on the same straight line;

[0006] The first mixed reality glasses construct a target coordinate system based on a first position according to a preset construction rule. The first position refers to the positions of the three marker points in a first world coordinate system, and the first world coordinate system is the world coordinate system of a first virtual scene displayed by the first mixed reality glasses;

[0007] The second mixed reality glasses construct the target coordinate system based on a second position according to the preset construction rule. The second position refers to the positions of the three marker points in a second world coordinate system, and the second world coordinate system is the world coordinate system of a second virtual scene displayed by the second mixed reality glasses;

[0008] The first mixed reality glasses determine a first pose of a target physical card played by a user wearing the first mixed reality glasses in the first world coordinate system, convert the first pose into a second pose in the target coordinate system, send the second pose to the second mixed reality glasses, and display a virtual animation corresponding to the target physical card in the first virtual scene according to the first pose;

[0009] The second mixed reality glasses receive the second pose, convert the second pose into a third pose in the second world coordinate system, and display the virtual animation corresponding to the card in the second virtual scene according to the third pose.

[0010] In a second aspect of the present application, a mixed reality system is provided, including: a first mixed reality glasses, a second mixed reality glasses, and three marker points arranged in a real scene, and the three marker points are not on the same straight line;

[0011] The first mixed reality glasses are configured to: construct a target coordinate system based on a first position according to a preset construction rule, where the first position refers to the positions of the three marker points in a first world coordinate system, and the first world coordinate system is the world coordinate system of a first virtual scene displayed by the first mixed reality glasses;

[0012] The second mixed reality glasses are configured to: construct the target coordinate system based on a second position according to the preset construction rule, where the second position refers to the positions of the three marker points in a second world coordinate system, and the second world coordinate system is the world coordinate system of a second virtual scene displayed by the second mixed reality glasses;

[0013] The first mixed reality glasses are further configured to: determine a first pose of a target entity card played by a user wearing the first mixed reality glasses in the first world coordinate system; convert the first pose into a second pose in the target coordinate system; send the second pose to the second mixed reality glasses; and display a virtual animation corresponding to the target entity card in the first virtual scene according to the first pose;

[0014] The second mixed reality glasses are further configured to: receive the second pose; convert the second pose into a third pose in the second world coordinate system; and display the virtual animation corresponding to the card in the second virtual scene according to the third pose.

[0015] In the technical solution provided by the embodiments of the present application, each mixed reality glasses in the mixed reality system identifies three marker points preset in the real scene, and then constructs a unified target coordinate system through these three identified marker points. That is to say, in the system, multiple mixed reality glasses can construct a unified target coordinate system based on these three marker points. In this way, when a player plays a card, the mixed reality glasses worn by the player identify the physical card played by the player, calculate the pose of the physical card in the unified constructed target coordinate system, and send the pose to the mixed reality glasses worn by other players. The mixed reality glasses of other players convert the pose into the pose in the world coordinate system of the virtual scene displayed by the local glasses, and display the virtual animation corresponding to the card in its virtual scene according to the converted pose. Displaying the virtual animation corresponding to the physical card according to the pose of the physical card can bring an immersive offline card table game experience to the player. Moreover, each player's mixed reality glasses only need to identify the physical card played by themselves, without identifying the physical cards played by other players, which can not only reduce the required computing power, but also avoid the problem that cards at a relatively long distance (such as the cards played by the opponent) cannot be recognized. It can be seen that this solution can bring an immersive offline card table game experience to the player with less computing power through a simple marker structure and MR glasses. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is a schematic flowchart of a display control method provided by an embodiment of the present application;

[0018] Figure 2 It is a schematic diagram of the setting of three marker points provided by an embodiment of the present application;

[0019] Figures 3 - 6 It is a schematic diagram of constructing a target coordinate system based on three marker points provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] To enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the scope of protection of this application.

[0021] In addition, in some processes described in the specification, claims, and the above-mentioned accompanying drawings of this application, there are multiple operations that appear in a specific order. These operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., do not represent a sequence, and do not limit that "first" and "second" are of different types.

[0022] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0023] First, the vocabulary involved in the embodiments of this application will be described. It can be understood that this description is for a clearer understanding of the embodiments of this application and does not necessarily constitute a limitation on the embodiments of this application.

[0024] Mixed Reality (MR) is a visualization environment that uses computer image technology, sensing technology, and visualization wearable technology to enable digital virtual objects and real-world objects to coexist, and enables users to build a feedback loop for interaction between the virtual and real worlds on the basis of normal perception of the real world, achieving timely and in-depth interaction between the virtual world and the real world.

[0025] Six degrees of freedom refers to the six degrees of freedom of a rigid body moving in three-dimensional space, that is, translation and rotation on three mutually perpendicular coordinate axes, namely the longitudinal axis (front - back), the transverse axis (left - right), and the vertical axis (up - down). These three axes are usually referenced by the direction in which the rigid body faces or travels, such as the three main axes of an aircraft.

[0026] iBeacon is a communication protocol based on Bluetooth Low Energy. It works in such a way that a Bluetooth Low Energy device equipped with the iBeacon protocol sends its unique ID to the surrounding area, and the application software that receives this ID will take some actions based on this ID. Therefore, the iBeacon device needs to interact with a device with Bluetooth Low Energy, and the interaction between the two is achieved through an APP.

[0027] Figure 1 It is a schematic flowchart of a display control method applied to a mixed reality system provided by an embodiment of this application.

[0028] Among them, the mixed reality system includes: a first mixed reality glasses, a second mixed reality glasses, and three marker points set in the real scene. The three marker points are not on the same straight line, that is to say, these three marker points can form a triangle.

[0029] It should be noted that the mixed reality system includes at least two mixed reality glasses, and the first mixed reality glasses and the second mixed reality glasses are any two mixed reality glasses in the mixed reality system. The information displayed on the screen of each mixed reality glasses is the fusion of the real scene and the virtual scene. Specifically, the mixed reality glasses enable the user to see an enhanced or modified real scene by superimposing virtual images on the user's field of vision, that is, a scene combining the virtual scene and the real scene.

[0030] In practical applications, the positions of these three marker points in the real scene can be set according to actual needs, and the embodiments of this application do not make specific limitations on this.

[0031] In some embodiments, the real scene includes a preset plane, and the three marker points are set on the preset plane.

[0032] Exemplarily, when multiple players (i.e., users) want to play a card game on the physical ground, these three marker points can be set on the ground.

[0033] Exemplarily, when multiple players want to play a card game on the physical tabletop, these three marker points can be set on the physical tabletop. That is to say, the preset plane is the physical tabletop in the real scene.

[0034] For example: These three marker points can be set in the edge area, corner position, and / or non-edge area of the physical tabletop. As Figure 2 shown in (a), marker point A and marker point C are set at the edge position of the physical tabletop, and marker point B is set at the corner position of the physical tabletop. As Figure 2 shown in (b), marker point A, marker point B, and marker point C are set in the non-edge area of the tabletop.

[0035] In some embodiments, in the real scenario, the three marked points are marked with different patterns. Among them, the pattern can be a digital pattern, a letter pattern, a QR code pattern, etc. The three marked points are marked with different patterns, which helps the mixed reality glasses to distinguish these three marked points.

[0036] As Figure 1 shown, the method includes the following steps:

[0037] 101. The first mixed reality glasses construct a target coordinate system based on the first position according to a preset construction rule.

[0038] Among them, the first position refers to the positions of the three marked points in the first world coordinate system, including: the positions of the three marked points in the first world coordinate system respectively.

[0039] Among them, the first world coordinate system is the world coordinate system of the first virtual scene displayed by the first mixed reality glasses.

[0040] The first world coordinate system is used to define the positions of objects in the entire first virtual scene.

[0041] The first mixed reality glasses determine the positions of the three marked points in the first world coordinate system to obtain the first position.

[0042] In some embodiments, the first mixed reality glasses can determine the positions of the three marked points in the first world coordinate system, that is, the first position, based on image recognition technology and 6DoF positioning technology. It should be noted that the specific implementation of determining the position of each marked point in the first world coordinate system based on image recognition technology and 6DoF positioning technology can refer to the prior art, and the embodiments of the present application do not make specific limitations on this.

[0043] In some embodiments, the three marked points are the first marked point, the second marked point, and the third marked point respectively. In the preset construction rule, the origin of the target coordinate system is the first marked point, the direction of the first axis of the target coordinate system is the direction from the first marked point to the second marked point, the direction of the second axis of the target coordinate system is obtained by rotating the first axis by 90° along the preset rotation direction in the plane where the three marked points are located, and the direction of the third axis of the target coordinate system is the cross product vector direction of the direction vector of the first axis and the direction vector of the second axis.

[0044] Optionally, the preset rotation direction can be the counterclockwise direction or the clockwise direction.

[0045] Next, how the mixed reality glasses distinguish these three marked points will be introduced.

[0046] In an alternative embodiment, the three marked points are marked with different patterns, that is, the patterns marked on any two marked points are different. In this way, the mixed reality glasses can distinguish the three marked points by recognizing the patterns.

[0047] In yet another alternative embodiment, as Figure 3 shown, the three marked points are marked point A, marked point B, and marked point C respectively. The pattern marked on marked point A is different from the patterns marked on marked point B and marked point C. Therefore, the mixed reality glasses can recognize marked point A as the first marked point by recognizing the pattern. Therefore, in the first world coordinate system, taking the first marked point (i.e., marked point A) as the origin of the target coordinate system, draw a line parallel to BC and passing through A, and rotate this line counterclockwise in the ABC plane to find the marked point that intersects the line first, for example Figure 3 shown as point B, and determine this marked point as the second marked point. In this way, the remaining marked point is the third marked point. In this embodiment, it is only necessary to ensure that the pattern marked on one of the three marked points (this marked point serves as the origin of the target coordinate system) is different from the patterns marked on the remaining two marked points, and the patterns marked on the remaining two marked points can be the same or different.

[0048] In an alternative embodiment, for the above 101, "the first mixed reality glasses construct a target coordinate system based on the first position according to a preset construction rule", the following steps can be used to implement it:

[0049] 1011. The first mixed reality glasses determine the position of the first marked point in the first world coordinate system as the origin of the target coordinate system.

[0050] The first mixed reality glasses can determine the first marked point from the three marked points according to the method provided above.

[0051] 1012. The first mixed reality glasses determine the direction from the position of the first marked point in the first world coordinate system to the position of the second marked point in the first world coordinate system as the direction of the first axis of the target coordinate system.

[0052] The first mixed reality glasses can determine the second marked point according to one of the above two methods.

[0053] 1013. The first mixed reality glasses determine the direction of the second axis of the target coordinate system according to the direction of the first axis.

[0054] Among them, the direction of the second axis is obtained by rotating the first axis 90 degrees in the target plane along the preset rotation direction, and the target plane is the plane where the positions of the three marked points in the first world coordinate system are located together.

[0055] Among them, the preset rotation direction can be clockwise or counterclockwise.

[0056] 1014. The first mixed reality glasses determine the direction of the cross product vector of the direction of the first axis and the second axis as the direction of the third axis of the target coordinate system.

[0057] The cross product vector is obtained by cross multiplying the direction vectors of the first axis and the second axis.

[0058] Based on the positions of the three marker points in the first world coordinate system, the direction vectors of the first axis and the second axis can be calculated, and then the direction vector of the third axis can be obtained by using cross product (cross). The matrix composed of these three unit vectors is the rotation matrix for the mutual conversion between the first world coordinate system and the target coordinate system. Combining with the origin coordinates of the target coordinate system, the translation amount can be determined. According to the rotation matrix and the translation amount, the conversion relationship between the first world coordinate system and the target coordinate system, that is, the conversion matrix, can be determined.

[0059] In the embodiment of the present application, the first mixed reality glasses construct the target coordinate system, that is, define the target coordinate system, that is, determine the conversion relationship between the first world coordinate system and the target coordinate system, such as the conversion matrix.

[0060] In some embodiments, the first mixed reality glasses determine the first conversion relationship between the first world coordinate system and the target coordinate system.

[0061] Next, the construction process of the target coordinate system will be introduced by way of Figures 3 - 6 exemplary. As Figure 3 shown, the three marker points are A, B, and C respectively. The mixed reality glasses determine the positions of points A, B, and C in the world coordinate system of its virtual scene through image recognition technology and 6DoF positioning technology. The three points can determine a plane ABC in the virtual scene or the world coordinate system. Among them, point A is the origin, and a straight line parallel to BC and passing through A is drawn. Rotate this straight line counterclockwise on the ABC plane to find the point among B or C that intersects the straight line first Figure 4 In the example shown, the point that intersects the straight line first is point B. The direction from A to B is used as the direction of the x-axis of the target coordinate system; rotate the x-axis (that is, the AB line) counterclockwise by 90 degrees on the ABC plane to find point D. Then the direction from A to D is used as the direction of the y-axis of the target coordinate system, as Figure 5 shown; calculate the vector cross product of AB and AD, and the direction of the z-axis of the target coordinate system can be obtained, Figure 6 which is the finally constructed target coordinate system.

[0062] In practical applications, the mixed reality glasses of all players can determine a unified target coordinate system through the above steps, and this target coordinate system can be called the game space coordinate system.

[0063] 102. The second mixed reality glasses construct the target coordinate system based on the second position according to the preset construction rules.

[0064] Among them, the second position refers to the positions of the three marker points in the second world coordinate system, and the second world coordinate system is the world coordinate system of the second virtual scene displayed by the second mixed reality glasses.

[0065] Among them, the second world coordinate system is the world coordinate system of the second virtual scene displayed by the second mixed reality glasses.

[0066] The second world coordinate system is used to define the positions of objects in the entire second virtual scene.

[0067] The second mixed reality glasses determine the positions of the three marker points in the second world coordinate system to obtain the second position.

[0068] In some embodiments, the second mixed reality glasses can determine the positions of the three marker points in the second world coordinate system, that is, the second position, based on image recognition technology and 6DoF positioning technology. It should be noted that the specific implementation of determining the position of each marker point in the second world coordinate system based on image recognition technology and 6DoF positioning technology can refer to the prior art, and the embodiments of the present application do not make specific limitations on this.

[0069] In an alternative implementation, for the above "the second mixed reality glasses construct the target coordinate system based on the second position according to the preset construction rules" in 102, the following steps can be adopted to implement:

[0070] 1021. The second mixed reality glasses determine the position of the first marker point in the second world coordinate system as the origin of the target coordinate system.

[0071] The second mixed reality glasses can determine the first marker point from the three marker points in the manner provided above.

[0072] 1022. The second mixed reality glasses determine the direction from the position of the first marker point in the second world coordinate system to the position of the second marker point in the second world coordinate system as the direction of the first axis of the target coordinate system.

[0073] The second mixed reality glasses can determine the second marker point in one of the above two ways.

[0074] 1023. The second mixed reality glasses determine the direction of the second axis of the target coordinate system according to the direction of the first axis.

[0075] Wherein, the direction of the second axis is obtained by rotating the first axis by 90 degrees in a preset rotation direction in the target plane, and the target plane is the plane where the positions of the three marked points in the second world coordinate system are located together.

[0076] Wherein, the preset rotation direction can be clockwise or counterclockwise.

[0077] 1024. The second mixed reality glasses determine the direction of the cross product vector of the direction of the first axis and the second axis as the direction of the third axis of the target coordinate system.

[0078] The cross product vector is obtained by cross multiplying the direction vector of the first axis and the direction vector of the second axis.

[0079] Based on the positions of the three marked points in the second world coordinate system, the direction vectors of the first axis and the second axis can be calculated, and then the direction vector of the third axis can be obtained by using cross product (cross). The matrix composed of these three unit vectors is the rotation matrix for the mutual conversion between the first world coordinate system and the target coordinate system. Combining with the origin coordinates of the target coordinate system, the translation amount can be determined. According to the rotation matrix and the translation amount, the conversion relationship between the first world coordinate system and the target coordinate system, that is, the conversion matrix, can be determined.

[0080] In the embodiments of the present application, the second mixed reality glasses construct the target coordinate system, that is, define the target coordinate system, that is, determine the conversion relationship between the second world coordinate system and the target coordinate system, such as the conversion matrix.

[0081] In some embodiments, the second mixed reality glasses determine the second conversion relationship between the second world coordinate system and the target coordinate system.

[0082] It should be noted that in practical applications, the first world coordinate system and the second world coordinate system are different coordinate systems.

[0083] Multiple mixed reality glasses in the mixed reality system can construct the same target coordinate system according to the same preset construction rules and the same three marked points. That is to say, multiple mixed reality glasses can construct a unified target coordinate system.

[0084] Multiple mixed reality glasses in the mixed reality system can default to use the same preset construction rules, or can negotiate to use the same preset construction rules.

[0085] 103. The first mixed reality glasses determine the first pose of the target entity card played by the user wearing the first mixed reality glasses in the first world coordinate system.

[0086] The first mixed reality glasses can determine the first pose of the target entity card played by the user wearing the first mixed reality glasses in the first world coordinate system based on image recognition technology and 6DoF positioning technology. It should be noted that the pose includes position and attitude.

[0087] 104. The first mixed reality glasses convert the first pose into the second pose in the target coordinate system and send the second pose to the second mixed reality glasses.

[0088] Among them, the first mixed reality glasses can convert the first pose into the second pose in the target coordinate system according to the first conversion relationship between the first world coordinate system and the target coordinate system.

[0089] In some embodiments, the first mixed reality glasses can directly send the second pose to other mixed reality glasses in the system, or send the second pose to the server, and then the server sends the second pose to other mixed reality glasses in the system.

[0090] 105. The first mixed reality glasses display the virtual animation corresponding to the target entity card in the first virtual scene according to the first pose.

[0091] According to the position in the first pose, determine the target position of the virtual animation in the first virtual scene; according to the attitude in the first pose, determine the target pose of the virtual animation in the first virtual scene, and display the virtual animation in the first virtual scene according to the target position and target pose.

[0092] In practical applications, different poses will result in different positions and / or orientations of the virtual animation.

[0093] It should be noted that the execution order of the above steps 104 and 105 is not specifically limited. For example: step 104 can be executed before step 105, can be executed after step 105, or can be executed simultaneously with step 105.

[0094] 106. The second mixed reality glasses receive the second pose.

[0095] 107. The second mixed reality glasses convert the second pose into the third pose in the second world coordinate system.

[0096] Wherein, the second mixed reality glasses can convert the second pose into a third pose in the second world coordinate system according to a second conversion relationship between the second world coordinate system and the target coordinate system.

[0097] 108. The second mixed reality glasses display a virtual animation corresponding to the card in the second virtual scene according to the third pose.

[0098] Wherein, the virtual animation includes a virtual character corresponding to the target entity card and / or a skill special effect released by the virtual character.

[0099] Determine the target position of the virtual animation in the second virtual scene according to the position in the third pose; determine the target pose of the virtual animation in the second virtual scene according to the pose in the third pose, and display the virtual animation in the second virtual scene according to the target position and the target pose.

[0100] In practical applications, different poses result in different positions and / or orientations of the virtual animation.

[0101] It should be noted that the pose of the virtual animation corresponding to the target entity card displayed by the first mixed reality glasses in the target coordinate system is the same as the pose of the virtual animation corresponding to the target entity card displayed by the second mixed reality glasses in the target coordinate system. However, since the positions and orientations of each player are different, the picture content of the virtual animation seen by each player on the screen of the mixed reality glasses they wear is different. For example, some players see the front of the virtual character corresponding to the target entity card, while some players see the back of the virtual character corresponding to the target entity card.

[0102] In some embodiments, the above method may further include the following steps:

[0103] 109. When the first mixed reality glasses recognize that the user wearing the first mixed reality glasses places the entity card in a preset area and the stationary duration of the entity card in the preset area reaches a preset duration, the first mixed reality glasses determine the entity card as the target entity card played by the user.

[0104] After the first mixed reality glasses recognize that the user places the entity card on the plane where the three marked points are located (such as the desktop mentioned above), timing can be performed. If the stationary duration of the entity card on the desktop reaches the preset duration, the entity card is determined as the target entity card played by the user, that is, the first mixed reality glasses recognize the card-playing action.

[0105] The size of the preset duration can be set according to actual needs, and the embodiments of the present application do not make specific limitations in this regard.

[0106] In practical applications, when the first mixed reality glasses recognize that the user wearing the first mixed reality glasses places a physical card in a preset area, the stationary duration of the physical card in the preset area reaches a preset duration, and the front side of the physical card faces upward, the first mixed reality glasses determine the physical card as the target physical card played by the user.

[0107] After determining that a card has been played, perform the above steps 103 to 108. In step 108, a virtual character can be displayed first, and then in response to the user's gesture instruction, control the virtual character to release a skill, that is, display the skill special effect released by the virtual character.

[0108] In this embodiment, this way of playing card interaction reduces the player's command actions, and the interaction process is also closer to the real way of playing cards, greatly improving the player experience.

[0109] In some embodiments, when the first mixed reality glasses respond to entering the first virtual scene, perform "construct a target coordinate system based on the first position according to the preset construction rules" in the above step 101.

[0110] In some embodiments, when the second mixed reality glasses respond to entering the second virtual scene, perform "construct the target coordinate system based on the second position according to the preset construction rules" in the above step 102.

[0111] In practical applications, when a player moves while wearing mixed reality glasses, due to the calculation error of 6Dof, there will be a deviation in the movement offset of the user's head in the virtual space and the real space. That is to say, as the deviation accumulates, the previously constructed target coordinate system will become invalid. Therefore, to solve this problem, each mixed reality glasses can perform the construction of the target coordinate system at preset time intervals, that is, update the definition of the target coordinate system or the conversion relationship between the world coordinate system of the virtual scene and the target coordinate system at preset time intervals.

[0112] That is: after the first mixed reality glasses enter the first virtual scene, at preset time intervals, perform the step of constructing a target coordinate system based on the first position according to the preset construction rules (that is, step 101); after the second mixed reality glasses enter the second virtual scene, at the preset time intervals, perform the step of constructing the target coordinate system based on the second position according to the preset construction rules (that is, step 102).

[0113] In some embodiments, the moment when the first mixed reality glasses enter the first virtual scene is the same as the moment when the second mixed reality glasses enter the second virtual scene.

[0114] In some embodiments, a first camera is provided on the first mixed reality glasses; the method further includes:

[0115] 110. The first mixed reality glasses acquire a current image frame collected by the first camera.

[0116] In practical applications, the first camera includes a left-eye camera and a right-eye camera, and the current image frame includes a current left image frame collected by the left-eye camera and a current right image frame collected by the right-eye camera.

[0117] 111. The first mixed reality glasses determine the first position according to the current image frame.

[0118] When three marker points can be recognized from the current image frame, the first position can be determined based on the current image frame by using image recognition technology and 6DoF positioning technology.

[0119] However, during the game process, it is often impossible to recognize some or all of the marker points from the current image frame due to reasons such as distance, light, and angle. Then, it is impossible to directly determine the first position based on the current image frame. In order to be able to update the target coordinate system even when the marker points are lost, positioning devices can be respectively set at the three marker points in the real scene. The positioning devices emit Bluetooth signals, and the first mixed reality glasses determine the distance between the marker points where the positioning devices are located and the head of the user wearing the first mixed reality glasses in the real scene according to the signal strength of the received Bluetooth signals.

[0120] In some embodiments, the positioning device can be an iBeacon device.

[0121] The above "the first mixed reality glasses determine the first position according to the current image frame" in 111 can be implemented by the following steps:

[0122] S11. When there is a first target marker point among the three marker points that cannot be recognized from the current image frame, the first mixed reality glasses determine the proportionality coefficient between the virtual distance between two points in the first virtual scene and the real distance between the two points in the real scene.

[0123] In an alternative implementation manner, S11 can be implemented by the following steps:

[0124] S111a. When there is a first target marker point among the three marker points that cannot be recognized from the current image frame and there is a second target marker point that can be recognized from the current image frame, the first mixed reality glasses determine the second virtual distance between the second target marker point and the head in the first virtual scene according to the current image frame.

[0125] Based on the current image frame, determine the current position of the second target marker point in the first world coordinate system, obtain the current position of the head in the first world coordinate system, and determine the second virtual distance between the second target marker point and the head in the first virtual scene according to the current position of the second target marker point in the first world coordinate system and the current position of the head in the first world coordinate system.

[0126] S112a. The first mixed reality glasses determine the proportionality coefficient according to the second virtual distance and the second real distance between the second target marker point and the head in the real scene at the current moment.

[0127] The ratio of the second virtual distance to the second real distance can be used as the proportionality coefficient.

[0128] In another alternative implementation manner, the following steps can be adopted to implement S11:

[0129] S111b. When there is no second target marker point that can be recognized from the current image frame among the three marker points, the first mixed reality glasses determine the third virtual distance between the third target marker point and the head in the first virtual scene at the second moment according to the second image frame collected at the second moment.

[0130] The third target marker point is a marker point that can be recognized from the second image frame, and the second image frame is the most recent image frame that can recognize any marker point among the three marker points.

[0131] Based on the second image frame, determine the position of the third target marker point at the second moment in the first world coordinate system, obtain the position of the head at the second moment in the first world coordinate system, and determine the third virtual distance between the second target marker point and the head in the first virtual scene at the second moment according to the position of the third target marker point at the second moment in the first world coordinate system and the position of the head at the second moment in the first world coordinate system.

[0132] S112b. The first mixed reality glasses determine the proportionality coefficient according to the third virtual distance and the third real distance between the third target marker point and the head in the real scene at the second moment.

[0133] The ratio of the third virtual distance to the third real distance can be used as the proportionality coefficient.

[0134] S12. The first mixed reality glasses determine a first virtual distance between the first target marker point and the head in the first virtual scene at the current moment according to the proportionality coefficient and a first real distance between the first target marker point and the head in the real scene at the current moment.

[0135] The product of the proportionality coefficient and the first real distance between the first target marker point and the head in the real scene at the current moment can be determined as the first virtual distance between the first target marker point and the head in the first virtual scene at the current moment.

[0136] S13. The first mixed reality glasses acquire a first image frame collected at a first moment.

[0137] Wherein, the target image frame is the most recent image frame that can recognize the first target marker point.

[0138] Wherein, the first moment is earlier than the current moment.

[0139] S14. The first mixed reality glasses determine a first position of the first target marker point in the first world coordinate system at the first moment according to the first image frame.

[0140] S15. The first mixed reality glasses determine a position of the first target marker point in the first world coordinate system at the current moment on the target straight line based on the second virtual distance.

[0141] Wherein, the target straight line is a straight line in the first virtual scene that passes through the first position and the position of the head in the first world coordinate at the current moment.

[0142] Two points can be determined on the target straight line, and the distances from both of these two points to the position of the head in the first world coordinate at the current moment are both the second virtual distance. From these two points, a target point that is closer to the position of the first target marker point recognized most recently can be determined, and the position of the target point is determined as the position of the first target marker point in the first world coordinate system at the current moment.

[0143] Optionally, if in the current image frame, marker point A and marker point B can be recognized, but point C cannot be recognized, then in the first world coordinate system, according to the current position of the user's head and the virtual distance from point C to the head estimated in step S12 at the current moment, a sphere surrounding the head is determined, and the intersection points of this sphere and the target circle are calculated. The target circle is: in the first world coordinate system, based on the positions of marker points A and C and the interior angles of triangle ABC, the possible positions of point C are determined, and the possible positions of point C form the target circle. When there is one intersection point between this sphere and the target circle, the position of the intersection point is taken as the current position of point C; when there are two intersection points between this sphere and the target circle, the position of the intersection point closer to the position of point C determined last time is taken as the current position of point C; when there are infinitely many or zero intersection points, the current position of point C is determined according to the above step S15. It should be noted that in this embodiment, all positions refer to the positions in the first world coordinate system.

[0144] The specific implementation of the above steps 110 and 111 is introduced by taking the first mixed reality glasses as an example. Other mixed reality glasses in the system can determine the positions of the three marker points in the world coordinate system of their virtual scenes according to the same method above.

[0145] In traditional card tabletop games, the game rules can only be explained or read from the manual by oneself, with a relatively high entry threshold. All the numerical calculations during the game process also have to be done by the players themselves, and there is no immersive experience during the game process. The mixed reality glasses provided by the embodiments of the present application can automatically explain the game rules to the user and calculate various numerical values during the game process, effectively reducing the entry threshold for the user. Moreover, through the technical solution provided by the embodiments of the present application, the player's physical cards can be visualized and real-time located. And during the game process, each player's mixed reality glasses only need to recognize the cards played by the player himself, without the need to recognize the cards of other players, greatly reducing the computing power consumption of the recognition algorithm and also avoiding the problem of difficult recognition of cards at a long distance. In this way, the game performance can be effectively improved.

[0146] An embodiment of the present application further provides a mixed reality system, which includes: a first mixed reality glasses, a second mixed reality glasses, and three marker points arranged in the real scene, and the three marker points are not on the same straight line.

[0147] Wherein,

[0148] The first mixed reality glasses are used to: construct a target coordinate system based on the first position according to a preset construction rule, where the first position refers to the positions of the three marker points in the first world coordinate system, and the first world coordinate system is the world coordinate system of the first virtual scene displayed by the first mixed reality glasses;

[0149] The second mixed reality glasses are configured to: construct the target coordinate system based on the second position according to the preset construction rules, where the second position refers to the positions of the three marked points in the second world coordinate system, and the second world coordinate system is the world coordinate system of the second virtual scene displayed by the second mixed reality glasses;

[0150] The first mixed reality glasses are further configured to: determine the first pose of the target entity card played by the user wearing the first mixed reality glasses in the first world coordinate system; convert the first pose into the second pose in the target coordinate system; send the second pose to the second mixed reality glasses; and display the virtual animation corresponding to the target entity card in the first virtual scene according to the first pose;

[0151] The second mixed reality glasses are further configured to: receive the second pose; convert the second pose into the third pose in the second world coordinate system; and display the virtual animation corresponding to the card in the second virtual scene according to the third pose.

[0152] It should be noted here that: for the content not detailed in the system provided in the embodiments of the present application, reference may be made to the corresponding content in the above embodiments, which will not be elaborated here. In addition, the system provided in the embodiments of the present application can not only implement the above functions, but also implement other functions in the above embodiments. For specific details, reference may be made to the corresponding content in the above embodiments, which will not be elaborated here.

[0153] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0154] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM (Read Only Memory), RAM (Random Access Memory), magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; each embodiment can be freely combined without conflict; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A display control method applied to a mixed reality system, characterized in that The mixed reality system includes: a first mixed reality glasses, a second mixed reality glasses, and three marker points set in the real scene, and the three marker points are not on the same straight line; The first mixed reality glasses construct a target coordinate system based on the first position according to a preset construction rule, where the first position refers to the positions of the three marker points in the first world coordinate system, and the first world coordinate system is the world coordinate system of the first virtual scene displayed by the first mixed reality glasses; The second mixed reality glasses construct the target coordinate system based on the second position according to the preset construction rule, where the second position refers to the positions of the three marker points in the second world coordinate system, and the second world coordinate system is the world coordinate system of the second virtual scene displayed by the second mixed reality glasses; The first mixed reality glasses determine the first pose of the target entity card played by the user wearing the first mixed reality glasses in the first world coordinate system, convert the first pose into the second pose in the target coordinate system, send the second pose to the second mixed reality glasses, and display the virtual animation corresponding to the target entity card in the first virtual scene according to the first pose; The second mixed reality glasses receive the second pose, convert the second pose into the third pose in the second world coordinate system, and display the virtual animation corresponding to the card in the second virtual scene according to the third pose.

2. The method according to claim 1, wherein The three marker points are respectively a first marker point, a second marker point, and a third marker point; In the preset construction rule, the origin of the target coordinate system is the first marker point, the direction of the first axis of the target coordinate system is the direction from the first marker point to the second marker point, the direction of the second axis of the target coordinate system is obtained by rotating the first axis 90° along a preset direction in the plane where the three marker points are located, and the third axis of the target coordinate system is the direction of the cross product vector of the direction vector of the first axis and the direction vector of the second axis.

3. The method according to claim 2, wherein The first mixed reality glasses construct a target coordinate system based on the first position according to a preset construction rule, including: The first mixed reality glasses determine the position of the first marker point in the first world coordinate system as the origin of the target coordinate system; The first mixed reality glasses determine the direction from the position of the first marker point in the first world coordinate system to the position of the second marker point in the first world coordinate system as the direction of the first axis of the target coordinate system; The first mixed reality glasses determine the direction of the second axis of the target coordinate system according to the direction of the first axis, where the direction of the second axis is obtained by rotating the first axis 90 degrees along a preset rotation direction in the target plane, and the target plane is the plane where the positions of the three marker points in the first world coordinate system are located together; The first mixed reality glasses determine the direction of the cross product vector of the first axis and the second axis as the direction of the third axis of the target coordinate system.

4. The method according to any one of claims 1 to 3, characterized in that The real scene includes a preset plane, and the three marking points are set on the preset plane.

5. The method according to claim 4, wherein The preset plane is an entity desktop in the real scene.

6. The method according to any one of claims 1 to 3, characterized in that, In the real scene, the three marking points are marked with different patterns.

7. The method according to any one of claims 1 to 3, characterized in that It further includes: The first mixed reality glasses determine a first conversion relationship between the first world coordinate system and the target coordinate system; The first mixed reality glasses convert the first pose into a second pose in the target coordinate system, including: The first mixed reality glasses convert the first pose into a second pose in the target coordinate system according to the first conversion relationship.

8. The method according to any one of claims 1 to 3, characterized in that, It further includes: The second mixed reality glasses determine a second conversion relationship between the second world coordinate system and the target coordinate system; The second mixed reality glasses convert the second pose into a third pose in the second world coordinate system, including: The second mixed reality glasses convert the second pose into a third pose in the second world coordinate system according to the second conversion relationship.

9. The method according to any one of claims 1 to 3, characterized in that It further includes: When the first mixed reality glasses recognize that the user wearing the first mixed reality glasses places an entity card in a preset area and the stationary duration of the entity card in the preset area reaches a preset duration, the first mixed reality glasses determine the entity card as the target entity card played by the user.

10. The method according to any one of claims 1 to 3, characterized in that The virtual animation includes the virtual character corresponding to the target entity card and / or the skill special effects released by the virtual character.

11. The method according to any one of claims 1 to 3, characterized in that It further includes: The first mixed reality glasses, in response to entering the first virtual scene, execute the step of constructing a target coordinate system based on the first position according to a preset construction rule; The second mixed reality glasses, in response to entering the second virtual scene, execute the step of constructing the target coordinate system based on the second position according to the preset construction rule.

12. The method according to any one of claims 1 to 3, characterized in that, It further includes: After the first mixed reality glasses enter the first virtual scene, at every preset time interval, execute the step of constructing a target coordinate system based on the first position according to a preset construction rule; After the second mixed reality glasses enter the second virtual scene, at every preset time interval, execute the step of constructing the target coordinate system based on the second position according to the preset construction rule.

13. The method according to claim 12, characterized in that, The moment when the first mixed reality glasses enter the first virtual scene is the same as the moment when the second mixed reality glasses enter the second virtual scene.

14. The method according to any one of claims 1 to 3, characterized in that, The first mixed reality glasses are provided with a first camera; The method further includes: The first mixed reality glasses obtain the current image frame collected by the first camera; The first mixed reality glasses determine the first position according to the current image frame.

15. The method according to claim 14, wherein Positioning devices are respectively arranged at the three marking points in the real scene. The positioning devices emit Bluetooth signals, and the first mixed reality glasses determine the distance between the marking point where the positioning device is located and the head of the user wearing the first mixed reality glasses in the real scene according to the signal strength of the received Bluetooth signal. The first mixed reality glasses determine the first position according to the current image frame, including: When there is a first target marker point among the three marker points that cannot be recognized from the current image frame, the first mixed reality glasses determine a proportionality coefficient between the virtual distance between two points in the first virtual scene and the real distance between the two points in the real scene; The first mixed reality glasses determine the first virtual distance between the first target marker point and the head in the first virtual scene at the current moment according to the proportionality coefficient and the first real distance between the first target marker point and the head in the real scene at the current moment; The first mixed reality glasses obtain a first image frame collected at a first moment, and the target image frame is the most recent image frame that can recognize the first target marker point; The first mixed reality glasses determine the first position of the first target marker point in the first world coordinate system at the first moment according to the first image frame; The first mixed reality glasses determine the position of the first target marker point in the first world coordinate system at the current moment on the target straight line based on the second virtual distance; the target straight line is a straight line in the first virtual scene that passes through the first position and the position of the head in the first world coordinate at the current moment.

16. The method according to claim 15, wherein When there is a first target marker point among the three marker points that cannot be recognized from the current image frame, the first mixed reality glasses determine a proportionality coefficient between the virtual distance between two points in the first virtual scene and the real distance between the two points in the real scene, including: When there is a first target marker point among the three marker points that cannot be recognized from the current image frame and there is a second target marker point that can be recognized from the current image frame, the first mixed reality glasses determine the second virtual distance between the second target marker point and the head in the first virtual scene according to the current image frame; The first mixed reality glasses determine the proportionality coefficient according to the second virtual distance and the second real distance between the second target marker point and the head in the real scene at the current moment.

17. The method according to claim 15, characterized in that, When there is a first target marker point among the three marker points that cannot be recognized from the current image frame, the first mixed reality glasses determine a proportionality coefficient between the virtual distance between two points in the first virtual scene and the real distance between the two points in the real scene, including: When there is no second target marker point that can be recognized from the current image frame among the three marker points, the first mixed reality glasses determine the third virtual distance between the third target marker point and the head in the first virtual scene at a second moment according to a second image frame collected at the second moment, where the third target marker point is a marker point that can be recognized from the second image frame, and the second image frame is the most recent image frame that can recognize any marker point among the three marker points; The first mixed reality glasses determine the proportionality coefficient according to the third virtual distance and the third real distance between the third target marker point and the head in the real scene at the second moment.

18. A mixed reality system, characterized in that, Comprising: A first mixed reality glasses, a second mixed reality glasses, and three marker points arranged in a real scene, and the three marker points are not on the same straight line; The first mixed reality glasses are configured to: construct a target coordinate system based on a first position according to a preset construction rule, where the first position refers to the positions of the three marker points in a first world coordinate system, and the first world coordinate system is the world coordinate system of a first virtual scene displayed by the first mixed reality glasses; The second mixed reality glasses are configured to: construct the target coordinate system based on a second position according to the preset construction rule, where the second position refers to the positions of the three marker points in a second world coordinate system, and the second world coordinate system is the world coordinate system of a second virtual scene displayed by the second mixed reality glasses; The first mixed reality glasses are further configured to: determine a first pose of a target entity card played by a user wearing the first mixed reality glasses in the first world coordinate system; convert the first pose into a second pose in the target coordinate system; send the second pose to the second mixed reality glasses; and display a virtual animation corresponding to the target entity card in the first virtual scene according to the first pose; The second mixed reality glasses are further configured to: receive the second pose; convert the second pose into a third pose in the second world coordinate system; and display the virtual animation corresponding to the card in the second virtual scene according to the third pose.

Images