Coordinate system synchronization method, mixed reality system and computer readable storage medium

By calculating the coordinate origin deviation and direction difference, synchronizing the coordinate systems of multiple mixed reality devices, the problem of inconsistent viewing angles between devices is solved and the sharing effect of virtual objects is achieved.

CN119024994APending Publication Date: 2024-11-26GOERTEK INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310597257.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-23
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In mixed reality devices, inconsistent coordinate systems between multiple devices lead to inconsistent viewing angles, affecting the sharing and interaction of virtual three-dimensional scenes.

Method used

By determining the position and shape information of the target object in different devices, calculating the coordinate origin deviation and direction difference, the coordinate system synchronization between devices is achieved, and the consistency of the display content is ensured.

Benefits of technology

The picture content between multiple mixed reality devices is achieved, the problem of inconsistent viewing angles is solved, and the sharing effect of virtual objects is achieved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119024994A_ABST
    Figure CN119024994A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of mixed reality equipment control, in particular to a coordinate system synchronization method, a mixed reality system and a computer readable storage medium. The method comprises the following steps: determining a first position of a target object in a first local coordinate system corresponding to first equipment and a second position of the target object in a second local coordinate system corresponding to second equipment; determining corresponding first shape information of the target object in the first equipment and corresponding second shape information of the target object in the second equipment; coordinate origin deviation is determined according to the first position and the second position, and direction difference is determined according to the first shape information and the second shape information; and determining synchronization information between the first local coordinate system and the second local coordinate system according to the coordinate origin deviation and the direction difference. The local coordinate systems of two different devices are synchronized, so that the projection contents seen by two sharing parties are kept consistent, and the problem that the viewing angles of multiple MR devices are not consistent is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of mixed reality device control technology, and in particular to a coordinate system synchronization method, a mixed reality system and a computer-readable storage medium. Background Art

[0002] MR (Mixed Reality) technology is a technology that introduces virtual scenes into real space and enables real-time interaction with the virtual scenes. By wearing MR devices, users can project virtual three-dimensional scenes into real space and interact with the virtual three-dimensional scenes through body movements such as hands and head.

[0003] In a related scheme, for a scenario where multiple MR devices at different positions share information, for example, a sharer uses an MR device to distribute shared information to other sharees wearing MR devices. Since the sharer and the sharee are located in different positions in space, the origin positions of the coordinate systems of the multiple MR devices are different, resulting in inconsistent viewing angles of the shared information between the sharer and the sharee, and between multiple sharees.

[0004] However, in some scenarios where the sharer wants the viewing angles of the shared information in each MR device to be completely consistent, for example, when the shared information is a 3D model, viewing from other angles may block the details of the 3D model that the sharer wants to display. In this case, a method for synchronizing the image content between multiple MR devices is needed to solve the problem of inconsistent viewing angles between multiple MR devices.

[0005] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of the present invention is to provide a method for synchronizing a coordinate system, aiming to solve the problem of inconsistent viewing angles among multiple MR devices.

[0007] To achieve the above object, the present invention provides a method for synchronizing a coordinate system, the method comprising:

[0008] Determine a first position of the target object in a first local coordinate system corresponding to the first device, and a second position of the target object in a second local coordinate system corresponding to the second device;

[0009] Determine first shape information corresponding to the target object in the first device, and second shape information corresponding to the target object in the second device;

[0010] determining a coordinate origin deviation according to the first position and the second position, and determining a direction difference according to the first shape information and the second shape information;

[0011] The synchronization information between the first local coordinate system and the second local coordinate system is determined according to the coordinate origin deviation and the direction difference.

[0012] Optionally, determining the direction difference according to the first shape information and the second shape information includes:

[0013] Rotating the second shape information based on a preset rotation increment and a preset rotation axis;

[0014] determining a shape similarity between the second shape information and the first shape information after each rotation;

[0015] When the shape similarity is greater than or equal to a preset first similarity threshold, determining the direction difference according to the preset rotation increment and the number of rotations;

[0016] Otherwise, return to the step of determining the shape similarity between the second shape information and the first shape information after each rotation.

[0017] Optionally, the step of determining the shape similarity between the second shape information and the first shape information after each rotation comprises:

[0018] determining a shape descriptor distance between the second shape information and the first shape information, and determining the shape similarity according to the shape descriptor distance; and / or,

[0019] The curvature proximity between the second shape information and the first shape information is determined, and the shape similarity is determined according to the curvature proximity.

[0020] Optionally, determining the direction difference according to the first shape information and the second shape information includes:

[0021] determining a target rotation amount according to a posture difference between the first shape information and the second shape information;

[0022] Based on the target rotation amount, rotating the second shape information around a preset rotation axis;

[0023] determining a shape similarity between the second shape information after rotation and the first shape information;

[0024] If the shape similarity is greater than or equal to a preset second similarity threshold, the direction difference is determined;

[0025] Otherwise, return to the step of determining the posture difference between the first shape information and the second shape information.

[0026] Optionally, after the step of determining synchronization information between the first local coordinate system and the second local coordinate system according to the coordinate origin deviation and the direction difference, the step further includes:

[0027] Determining a second display posture of the object to be projected in the second local coordinate system according to the synchronization information and a first display posture of the object to be projected in the first local coordinate system;

[0028] According to the second display posture and the rendering information corresponding to the object to be projected, the object to be projected is projected on the display screen of the second device, so that the display contents between the second device and the first device are the same.

[0029] Optionally, the step of determining a second display posture of the object to be projected in the second local coordinate system according to the synchronization information and the first display posture of the object to be projected in the first local coordinate system includes:

[0030] Based on the synchronization information, determine the coordinate value of the first display posture in the first local coordinate system, and map it to the corresponding target mapping coordinate in the second local coordinate system;

[0031] Based on the target mapping coordinates, determining a projection position and a projection angle of the object to be projected in the second local coordinate system;

[0032] The second display posture is determined according to the projection position and the projection angle.

[0033] Optionally, before the step of determining a first position of the target object in a first local coordinate system corresponding to the first device and a second position of the target object in a second local coordinate system corresponding to the second device, the step further includes:

[0034] Extracting object feature information of a target reference object selected in the first image captured by the first device;

[0035] Determining feature similarity between each object in the second image captured by the second device and the feature information of the object;

[0036] The object whose feature similarity is greater than a preset feature similarity threshold is determined as the target object.

[0037] Optionally, before the step of determining the coordinate origin deviation according to the first position and the second position, and determining the direction difference according to the first shape information and the second shape information, the method further includes:

[0038] Acquire three-dimensional scene information obtained by scanning the action space by the first device;

[0039] Generating a virtual coordinate system at a preset position of the three-dimensional scene information as a global coordinate system;

[0040] The steps of determining the coordinate origin deviation according to the first position and the second position, and determining the direction difference according to the first shape information and the second shape information include:

[0041] Based on the global coordinate system, the steps of determining the coordinate origin deviation according to the first position and the second position, and determining the direction difference according to the first shape information and the second shape information are performed.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also provides a mixed reality system, which includes: at least two devices, a server, a memory, a processor, and a coordinate system synchronization program stored in the memory and executable on the processor, and the coordinate system synchronization program, when executed by the processor, implements the steps of the coordinate system synchronization method as described above.

[0043] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a coordinate system synchronization program is stored, and when the coordinate system synchronization program is executed by a processor, the steps of the coordinate system synchronization method as described above are implemented.

[0044] The embodiments of the present invention provide a coordinate system synchronization method, a mixed reality system and a computer-readable storage medium. By synchronizing the local coordinate systems between two different devices, the coordinate systems between the sharing party and the shared party are kept consistent, so that the projection content seen on the display screen of the shared party's device is consistent with the projection content seen on the display screen of the sharing party's device, thereby achieving the effect of sharing virtual objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of the architecture of the hardware operating space of the mixed reality system involved in an embodiment of the present invention;

[0046] Figure 2 It is a schematic flow chart of a first embodiment of a method for synchronizing a coordinate system of the present invention;

[0047] Figure 3 It is a flow chart of a second embodiment of the method for synchronizing a coordinate system of the present invention;

[0048] Figure 4 It is a flowchart diagram of the third embodiment of the coordinate system synchronization method of the present invention.

[0049] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0050] This application synchronizes the local coordinate systems between two different devices so that the coordinate systems between the sharing party and the shared party remain consistent, so that the projection content seen on the display screen of the shared party's device is consistent with the projection content seen on the display screen of the sharing party's device, achieving the effect of sharing virtual objects.

[0051] In order to better understand the above technical solution, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0052] As an implementation plan, Figure 1 It is a schematic diagram of the architecture of the hardware operating space of the mixed reality system involved in the embodiment of the present invention.

[0053] like Figure 1 As shown, the mixed reality system may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0054] Those skilled in the art will understand that Figure 1 The mixed reality system architecture shown in the figure does not constitute a limitation of the mixed reality system, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0055] like Figure 1As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a synchronization program of the coordinate system. Among them, the operating system is a program for managing and controlling the hardware and software resources of the mixed reality system, the synchronization program of the coordinate system, and the operation of other software or programs.

[0056] exist Figure 1 In the mixed reality system shown, the user interface 1003 is mainly used to connect to the terminal and communicate data with the terminal; the network interface 1004 is mainly used for the background server and communicates data with the background server; the processor 1001 can be used to call the synchronization program of the coordinate system stored in the memory 1005.

[0057] In this embodiment, the mixed reality system includes: a memory 1005, a processor 1001, and a coordinate system synchronization program stored in the memory and executable on the processor, wherein:

[0058] When the processor 1001 calls the synchronization program of the coordinate system stored in the memory 1005, the following operations are performed:

[0059] Determine a first position of the target object in a first local coordinate system corresponding to the first device, and a second position of the target object in a second local coordinate system corresponding to the second device;

[0060] Determine first shape information corresponding to the target object in the first device, and second shape information corresponding to the target object in the second device;

[0061] determining a coordinate origin deviation according to the first position and the second position, and determining a direction difference according to the first shape information and the second shape information;

[0062] The synchronization information between the first local coordinate system and the second local coordinate system is determined according to the coordinate origin deviation and the direction difference.

[0063] When the processor 1001 calls the synchronization program of the coordinate system stored in the memory 1005, the following operations are performed:

[0064] Rotating the second shape information based on a preset rotation increment and a preset rotation axis;

[0065] determining a shape similarity between the second shape information and the first shape information after each rotation;

[0066] When the shape similarity is greater than or equal to a preset first similarity threshold, determining the direction difference according to the preset rotation increment and the number of rotations;

[0067] Otherwise, return to the step of determining the shape similarity between the second shape information and the first shape information after each rotation.

[0068] When the processor 1001 calls the synchronization program of the coordinate system stored in the memory 1005, the following operations are performed:

[0069] determining a shape descriptor distance between the second shape information and the first shape information, and determining the shape similarity according to the shape descriptor distance; and / or,

[0070] The curvature proximity between the second shape information and the first shape information is determined, and the shape similarity is determined according to the curvature proximity.

[0071] When the processor 1001 calls the synchronization program of the coordinate system stored in the memory 1005, the following operations are performed:

[0072] determining a target rotation amount according to a posture difference between the first shape information and the second shape information;

[0073] Based on the target rotation amount, rotating the second shape information around a preset rotation axis;

[0074] determining a shape similarity between the second shape information after rotation and the first shape information;

[0075] If the shape similarity is greater than or equal to a preset second similarity threshold, the direction difference is determined;

[0076] Otherwise, return to the step of determining the posture difference between the first shape information and the second shape information.

[0077] When the processor 1001 calls the synchronization program of the coordinate system stored in the memory 1005, the following operations are performed:

[0078] Determining a second display posture of the object to be projected in the second local coordinate system according to the synchronization information and a first display posture of the object to be projected in the first local coordinate system;

[0079] According to the second display posture and the rendering information corresponding to the object to be projected, the object to be projected is projected on the display screen of the second device, so that the display contents between the second device and the first device are the same.

[0080] When the processor 1001 calls the synchronization program of the coordinate system stored in the memory 1005, the following operations are performed:

[0081] Based on the synchronization information, determine the coordinate value of the first display posture in the first local coordinate system, and map it to the corresponding target mapping coordinate in the second local coordinate system;

[0082] Based on the target mapping coordinates, determining a projection position and a projection angle of the object to be projected in the second local coordinate system;

[0083] The second display posture is determined according to the projection position and the projection angle.

[0084] When the processor 1001 calls the synchronization program of the coordinate system stored in the memory 1005, the following operations are performed:

[0085] Extracting object feature information of a target reference object selected in the first image captured by the first device;

[0086] Determining feature similarity between each object in the second image captured by the second device and the feature information of the object;

[0087] The object whose feature similarity is greater than a preset feature similarity threshold is determined as the target object.

[0088] When the processor 1001 calls the synchronization program of the coordinate system stored in the memory 1005, the following operations are performed:

[0089] Acquire three-dimensional scene information obtained by scanning the action space by the first device;

[0090] Generating a virtual coordinate system at a preset position of the three-dimensional scene information as a global coordinate system;

[0091] The steps of determining the coordinate origin deviation according to the first position and the second position, and determining the direction difference according to the first shape information and the second shape information include:

[0092] Based on the global coordinate system, the steps of determining the coordinate origin deviation according to the first position and the second position, and determining the direction difference according to the first shape information and the second shape information are performed.

[0093] Based on the hardware architecture of the mixed reality system based on the mixed reality device control technology, an embodiment of the coordinate system synchronization method of the present invention is proposed.

[0094] Reference Figure 2 In a first embodiment, the method for synchronizing the coordinate system comprises the following steps:

[0095] Step S10, determining a first position of the target object in a first local coordinate system corresponding to the first device, and a second position of the target object in a second local coordinate system corresponding to the second device;

[0096] In this embodiment, the first device and the second device are both wearable MR devices, and the first device and the second device are at different positions in the action space by default in this embodiment. After the user wears the MR device on the body and starts it, the first device and the second device will collect the target object that actually exists in the real space, and based on the local coordinate systems of the two devices themselves, generate the relative position information corresponding to the target object in their own local coordinate systems, that is, the first position and the second position, wherein the local coordinate system of the first device is the first local coordinate system, and the second local coordinate system is the same.

[0097] The local coordinate system is characterized as a local coordinate system of the device. As the position of the device changes, the origin position of the coordinate system will also change accordingly. Because the first device and the second device are located at different positions, the origin positions of the first local coordinate system and the second local coordinate system are also different. In this embodiment, the purpose of determining the first position and the second position of the target object is to determine the deviation of the origin position of the coordinate system of the first local coordinate system and the second local coordinate system, that is, the relative position information between the first device and the second device.

[0098] Optionally, in some embodiments, the position of the target object in the local coordinate system corresponding to the device can be determined by collecting one or more images of the target object by a visual camera system mounted on the device, and then calculating the distance, direction, and other information representing the position from the device as the origin to the target object through the images.

[0099] Optionally, in other implementation schemes, the laser sensor mounted on the device may scan objects in the surrounding environment to generate point cloud data, and the distance, direction, and other information representing the position from the laser sensor of the device as the origin to the target object may be calculated based on the point cloud data.

[0100] Step S20, determining first shape information corresponding to the target object in the first device, and second shape information corresponding to the target object in the second device;

[0101] In this embodiment, the shape information is characterized by the shape and size of the target object collected by the device at an angle corresponding to the current position. In this embodiment, the purpose of determining the first shape information and the second shape information of the target object is to determine the coordinate system direction deviation of the first local coordinate system and the second local coordinate system, that is, to determine the orientation of the display screen between the first device and the second device.

[0102] Optionally, in some embodiments, the shape information of the target object in the local coordinate system corresponding to the device can be determined by acquiring one or more images of the target object through a visual camera system mounted on the device, and then performing three-body modeling of the target object based on the images to obtain the shape information of the target object.

[0103] Optionally, in other implementation schemes, the laser sensor mounted on the device may scan objects in the surrounding environment to generate point cloud data, and the features of the target object in the point cloud data may be extracted to obtain the shape information of the target object.

[0104] Step S30, determining a coordinate origin deviation according to the first position and the second position, and determining a direction difference according to the first shape information and the second shape information;

[0105] In this embodiment, the coordinate origin deviation between the first local coordinate system and the second local coordinate system is determined based on the position information between the first position and the second position, and the coordinate system direction difference between the first local coordinate system and the second local coordinate system is determined based on the first shape information and the second shape information.

[0106] Optionally, in some embodiments, the height difference between the first position and the second position in the vertical direction may be compared first, and then the horizontal position difference between the first position and the second position may be compared, and the coordinate origin deviation may be determined based on the height difference and / or the horizontal position difference.

[0107] Optionally, in other embodiments, the horizontal position difference between the first position and the second position in the horizontal direction may be compared first, and then the height difference between the first position and the second position in the vertical direction may be compared, and the coordinate origin deviation may be determined based on the height difference and / or the horizontal position difference.

[0108] Optionally, in other embodiments, the horizontal position difference between the first position and the second position in the horizontal direction and the height difference between the first position and the second position in the vertical direction may be compared simultaneously, and the coordinate origin deviation may be determined based on the height difference and / or the horizontal position difference.

[0109] It should be noted that if there is a height difference only in the vertical direction between the first local coordinate system and the second local coordinate system, the height difference will be used as the coordinate origin deviation; if there is a horizontal position difference only in the horizontal direction between the first local coordinate system and the second local coordinate system, the horizontal position difference will be used as the coordinate origin deviation; if there are deviations in both the vertical and horizontal directions, the coordinate origin deviation will be determined based on the height difference and the horizontal position difference.

[0110] In this embodiment, the first shape information is used as a reference to determine the difference between the second shape information and the first shape information, and to determine the coordinate system direction deviation, that is, the direction difference, between the first local coordinate system and the second local coordinate system, that is, the direction difference, that is, to determine the direction difference of the display screen orientation of the second device relative to the first device.

[0111] Optionally, in some embodiments, how to determine the direction deviation based on the first shape information and the second shape information can be to rotate the second shape information around a preset rotation Z axis center point, and each time a preset rotation increment is rotated, the first shape information is compared with the second shape information, and the shape similarity between the second shape information after each rotation and the first shape information is determined, and the number of rotations is recorded. When the shape similarity obtained is greater than or equal to the preset first similarity threshold, the number of rotations and the preset rotation increment are multiplied to obtain the direction difference. Exemplarily, the preset rotation increment can be 0.1 degrees.

[0112] Furthermore, in the above-mentioned embodiment, the determination of shape similarity can be obtained by calculating the shape descriptor distance and / or curvature proximity. The shape descriptor distance refers to the Euclidean distance or other distance measurement between the features of two shapes. Curvature proximity refers to whether the curvatures of two shapes are close. Specifically, firstly, the features of the shape information are extracted through contour extraction, corner point detection, and edge detection algorithms, including but not limited to curvature, straight line segment length, angle, etc., and then the shape descriptor distance and curvature proximity are calculated for these features. Among them, by comparing the shape descriptor distance and curvature proximity, we can determine the shape similarity. If the shape descriptor distance and curvature proximity of the two shapes are very small, the shape similarity between them is very high. On the contrary, if these indicators are very large, the similarity between them is very low.

[0113] Optionally, in other embodiments, as to how to determine the direction deviation based on the first shape information and the second shape information, it is also possible to directly calculate the position difference between the first shape information and the second shape information, determine the target rotation amount of the second shape information, directly rotate around a preset center point of the rotation Z axis according to the target rotation amount, and then compare the rotated second shape information with the first shape information to determine the shape similarity. If the obtained shape similarity is greater than or equal to the preset second similarity threshold, it is determined that the rotation amount obtained based on the position difference is correct; if it is less than, the position difference between the first shape information and the second shape information in the current state is re-determined, and the subsequent rotation steps are performed again.

[0114] Furthermore, in the above embodiment, to determine the target rotation amount, the first shape information and the second shape information can be first converted into a point set in a coordinate system, and the corresponding points can be matched. Then, based on the point set matching result, the posture difference between the first shape information and the second shape information is calculated, where the posture difference can be expressed as a translation vector and a rotation matrix. Then, the rotation angle, that is, the target rotation amount, is extracted from the rotation matrix. The rotation matrix can be represented by Euler angles or quaternions, and the rotation angle can be extracted from it.

[0115] Exemplarily, assuming that the rotation matrix is ​​represented by Euler angles, the target rotation amount can be calculated using the following formula: target rotation amount = arccos ((tr (R) - 1) / 2);

[0116] where R is the rotation matrix and tr(R) is the trace of R (i.e., the sum of the diagonal elements).

[0117] It should be noted that the selection of the above implementation modes can be made according to the computing power of the device, and is not limited in this embodiment.

[0118] Step S40: determining synchronization information between the first local coordinate system and the second local coordinate system according to the coordinate origin deviation and the direction difference.

[0119] In this embodiment, after determining the coordinate origin deviation and the direction difference, synchronization information between the first local coordinate system and the second local coordinate system is determined according to the coordinate origin deviation and the direction difference. The synchronization information is characterized as information required when the projection content of the first device in the first local coordinate system is synchronized to the second local coordinate system.

[0120] After the projection content is synchronized between the two local coordinate systems through the synchronization information, the screen content displayed between the first device and the second device is consistent, wherein the screen content includes the size, shape, and field of view angle of the projection content.

[0121] In the technical solution provided in this embodiment, the synchronization information between the first local coordinate system and the second local coordinate system is determined by the coordinate origin deviation and the direction difference, so that the coordinate systems between the sharing party and the shared party are kept consistent, so that the projection content seen on the display screen of the shared party's device maintains the same size, shape, and field of view angle as the projection content seen on the display screen of the sharing party's device, thereby achieving the effect of sharing virtual objects.

[0122] Furthermore, in this embodiment, after step S40, the following steps are further included:

[0123] Step S50, determining a second display posture of the object to be projected in the second local coordinate system according to the synchronization information and the first display posture of the object to be projected in the first local coordinate system;

[0124] Step S60: projecting the object to be projected on the display screen of the second device according to the second display posture and the rendering information corresponding to the object to be projected, so that the display contents of the second device and the first device are the same.

[0125] Optionally, in this embodiment, assuming that the first device is the sharing party and the second device is the shared party, after the user of the first device displays a virtual three-dimensional image on its display screen, the three-dimensional image is shared to the second device as an object to be projected to the second device for display.

[0126] First, according to the synchronization information and the first display posture of the object to be projected in the first local coordinate system, the second display posture corresponding to the object to be projected in the second local coordinate system is determined. Among them, the display posture includes the display position and display angle of the object to be projected. Based on the synchronization information, a mapping matrix between the first local coordinate system and the second local coordinate system can be obtained. Based on the mapping matrix, the coordinate value of the object to be projected in the first local coordinate system and the corresponding target mapping coordinates in the second local coordinate system are determined. Based on the target mapping coordinates, the projection position and projection angle of the object to be projected in the second local coordinate system are determined, and the second display posture is determined according to the projection position and projection angle.

[0127] Then, according to the determined second display posture, the projection position and projection angle of the object to be projected in the second local coordinate system are determined, and then according to the rendering information corresponding to the object to be projected, the object to be projected is displayed at the projection position and the projection angle, so that the display content between the second device and the first device is the same, thereby realizing the sharing of virtual objects.

[0128] Reference Figure 3 In the second embodiment, based on the first embodiment, before step S10, the following steps are further included:

[0129] Step S70, extracting object feature information of the target reference object selected in the first image captured by the first device;

[0130] Step S80, determining a feature similarity between each object in the second image captured by the second device and the feature information of the object;

[0131] Step S90: determining the object whose feature similarity is greater than a preset feature similarity threshold as the target object.

[0132] Optionally, in this embodiment, after the device is started, the first device as the sharing party will capture images of the current surrounding environment to obtain one or more first images, and identify the mesh information from the first image and the position and object feature information of all object information based on the local coordinate system of the first device and send them to the server. The server will receive the mesh information and the identified object position and shape information sent by different devices in the same environment, and the server will randomly select a target reference object from all objects submitted by the first device, and then search for the object submitted by the second device based on the object feature information of the target reference object, and determine the object whose feature similarity obtained after the search is greater than the similarity threshold as the target object that can be captured by both the first device and the second device.

[0133] In addition, if there are multiple objects with a degree greater than the similarity threshold, the object with the largest feature similarity among the multiple objects is selected as the target object.

[0134] In the technical solution provided in this embodiment, in order to ensure that the target object determined in the second device and the second device is the same object, the server selects the target reference object from the first image sent by the first device, and then identifies the object closest to the target reference object from the second image sent by the second device as the target object that can be captured by the first device and the second device at the same time, thereby ensuring that the subsequent coordinate system synchronization process of the two devices is based on the same target object, thereby improving the accuracy of coordinate system synchronization.

[0135] Reference Figure 4 In a third embodiment, based on any embodiment, before step S30, the method further includes:

[0136] Step S100, obtaining three-dimensional scene information obtained by scanning the action space by the first device;

[0137] Step S110, generating a virtual coordinate system at a preset position of the three-dimensional scene information as a global coordinate system;

[0138] The step S30 comprises:

[0139] Step S31, based on the global coordinate system, executing the steps of determining the coordinate origin deviation according to the first position and the second position, and determining the direction difference according to the first shape information and the second shape information.

[0140] Optionally, in this embodiment, in order to facilitate the determination of the coordinate origin deviation and direction difference between two local coordinate systems, a global coordinate system is also introduced in this embodiment. The global coordinate system is characterized as the coordinate system in the space where the first device and the second device are located. The global coordinate system will not change with the movement of the first device and / or the second device.

[0141] Optionally, as to how to generate a virtual coordinate system at a preset position of the three-dimensional scene information, as a global coordinate system, a certain point in the actual environment is taken as the origin, a certain direction in the space is taken as the positive direction of the X-axis, the vertical upward direction is taken as the positive direction of the Y-axis, and a left-handed coordinate system is used to determine the positive direction of the Z-axis.

[0142] For example, two small balls can be generated at the preset positions of the three-dimensional scene information. The positions of the small balls in the global coordinate system are fixed and will not change with the movement of the device. A global coordinate system is constructed by the two small balls. Let small ball A and small ball B, with the center point of small ball A as the coordinate origin, the vertical direction as the positive direction of the Y axis, and the horizontal direction of small ball B relative to small ball A as the positive direction of the X axis. Using the left-handed coordinate system, we can determine the positive direction of the Z axis. In this way, we have constructed a global coordinate system based on these two small balls.

[0143] After the global coordinate system is constructed, both the coordinate origin deviation and the direction difference between the two device coordinate systems can be calculated based on the global coordinate system.

[0144] In the technical solution provided in this embodiment, a global coordinate system is constructed by using the three-dimensional scene information obtained by scanning the action space by the first device, so that the coordinate origin deviation and direction difference between the two device coordinate systems can be calculated based on the global coordinate system, thereby realizing simple and high-accuracy virtual object sharing.

[0145] In addition, it can be understood by a person skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the mixed reality system to implement the process steps of the embodiment of the above method.

[0146] Therefore, the present invention also provides a computer-readable storage medium, which stores a coordinate system synchronization program. When the coordinate system synchronization program is executed by a processor, the various steps of the coordinate system synchronization method described in the above embodiment are implemented.

[0147] The computer-readable storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which may store program codes.

[0148] It should be noted that since the storage medium provided in the embodiment of the present application is the storage medium used to implement the method of the embodiment of the present application, based on the method introduced in the embodiment of the present application, the person skilled in the art can understand the specific structure and deformation of the storage medium, so it is not repeated here. All storage media used in the method of the embodiment of the present application belong to the scope of protection of this application.

[0149] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0151] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0152] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0153] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0154] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0155] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for synchronizing a coordinate system, characterized in that: The coordinate system synchronization method includes: Determine a first position of the target object in a first local coordinate system corresponding to the first device, and a second position of the target object in a second local coordinate system corresponding to the second device; Determine first shape information corresponding to the target object in the first device, and second shape information corresponding to the target object in the second device; determining a coordinate origin deviation according to the first position and the second position, and determining a direction difference according to the first shape information and the second shape information; The synchronization information between the first local coordinate system and the second local coordinate system is determined according to the coordinate origin deviation and the direction difference.

2. The method for synchronizing a coordinate system according to claim 1, wherein: Determining the direction difference according to the first shape information and the second shape information comprises: Rotating the second shape information based on a preset rotation increment and a preset rotation axis; determining a shape similarity between the second shape information and the first shape information after each rotation; When the shape similarity is greater than or equal to a preset first similarity threshold, determining the direction difference according to the preset rotation increment and the number of rotations; Otherwise, return to the step of determining the shape similarity between the second shape information and the first shape information after each rotation.

3. The method for synchronizing a coordinate system as claimed in claim 2, characterized in that: The step of determining the shape similarity between the second shape information and the first shape information after each rotation comprises: determining a shape descriptor distance between the second shape information and the first shape information, and determining the shape similarity according to the shape descriptor distance; and / or, The curvature proximity between the second shape information and the first shape information is determined, and the shape similarity is determined according to the curvature proximity.

4. The method for synchronizing a coordinate system according to claim 1, wherein: Determining the direction difference according to the first shape information and the second shape information comprises: determining a target rotation amount according to a posture difference between the first shape information and the second shape information; Based on the target rotation amount, rotating the second shape information around a preset rotation axis; determining a shape similarity between the second shape information after rotation and the first shape information; If the shape similarity is greater than or equal to a preset second similarity threshold, the direction difference is determined; Otherwise, return to the step of determining the posture difference between the first shape information and the second shape information.

5. The method for synchronizing a coordinate system according to claim 1, wherein: After the step of determining synchronization information between the first local coordinate system and the second local coordinate system according to the coordinate origin deviation and the direction difference, the method further includes: Determining a second display posture of the object to be projected in the second local coordinate system according to the synchronization information and a first display posture of the object to be projected in the first local coordinate system; According to the second display posture and the rendering information corresponding to the object to be projected, the object to be projected is projected on the display screen of the second device, so that the display contents between the second device and the first device are the same.

6. The method for synchronizing a coordinate system as claimed in claim 5, characterized in that: The step of determining a second display posture of the object to be projected in the second local coordinate system according to the synchronization information and the first display posture of the object to be projected in the first local coordinate system comprises: Based on the synchronization information, determine the coordinate value of the first display posture in the first local coordinate system, and map it to the corresponding target mapping coordinate in the second local coordinate system; Based on the target mapping coordinates, determining a projection position and a projection angle of the object to be projected in the second local coordinate system; The second display posture is determined according to the projection position and the projection angle.

7. The method for synchronizing a coordinate system according to claim 1, wherein: Before the step of determining a first position of the target object in a first local coordinate system corresponding to the first device and a second position of the target object in a second local coordinate system corresponding to the second device, the step further includes: Extracting object feature information of a target reference object selected in the first image captured by the first device; Determining feature similarity between each object in the second image captured by the second device and the feature information of the object; The object whose feature similarity is greater than a preset feature similarity threshold is determined as the target object.

8. The method for synchronizing a coordinate system according to claim 1, wherein: Before the steps of determining the coordinate origin deviation according to the first position and the second position, and determining the direction difference according to the first shape information and the second shape information, the method further includes: Acquire three-dimensional scene information obtained by scanning the action space by the first device; Generating a virtual coordinate system at a preset position of the three-dimensional scene information as a global coordinate system; The steps of determining the coordinate origin deviation according to the first position and the second position, and determining the direction difference according to the first shape information and the second shape information include: Based on the global coordinate system, the steps of determining the coordinate origin deviation according to the first position and the second position, and determining the direction difference according to the first shape information and the second shape information are performed.

9. A mixed reality system, characterized in that: The mixed reality system includes: at least two devices, a server, a memory, a processor, and a coordinate system synchronization program stored in the memory and executable on the processor. When the coordinate system synchronization program is executed by the processor, the steps of the coordinate system synchronization method described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a coordinate system synchronization program, and when the coordinate system synchronization program is executed by a processor, the steps of the coordinate system synchronization method according to any one of claims 1 to 8 are implemented.