Space anchor point sharing method, head-mounted display device and computer readable medium

Abstract

The embodiment of the invention discloses a space anchor point sharing method, head-mounted display equipment and a computer readable medium. A specific implementation mode of the method comprises the steps that according to acquired space anchor point data information, space anchor point position and posture information of a to-be-generated space anchor point in a current scene is generated, and the current scene is displayed in a head-mounted display device; generating a space anchor point corresponding to the position and attitude information of the space anchor point according to the position and attitude information of the space anchor point; and in response to determining that the associated scene object included in the space anchor point data information exists in the current scene, performing association processing on the space anchor point and the associated scene object included in the space anchor point data information so as to share the space anchor point included in the space anchor point data information. According to the embodiment, synchronous movement of the shared space anchor point and the corresponding real object can be realized, and the user experience is further improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of virtual vision technology, and in particular to a spatial anchor point sharing method, a head-mounted display device, and a computer-readable medium. Background Art

[0002] Sharing of spatial anchors can enable spatial anchors to be used between different applications, devices or users, thereby improving the user experience. Currently, when sharing spatial anchors, the method usually adopted is: after determining the position of the shared spatial anchor in the current scene, the spatial anchor is shared.

[0003] However, in practice, it is found that when the commonly used method is used to share spatial anchor points, the following technical problems often occur:

[0004] Only the position of the shared spatial anchor point in the current scene is determined, and the shared spatial anchor point is not bound to the corresponding real object in the current scene. As a result, there is no fixed posture relationship between the shared spatial anchor point and the corresponding real object in the current scene. The shared spatial anchor point cannot move synchronously with the corresponding real object, resulting in a poor user experience.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose a spatial anchor point sharing method, a head-mounted display device, and a computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a spatial anchor point sharing method, which is applied to a head-mounted display device, comprising: generating spatial anchor point pose information of a to-be-generated spatial anchor point in a current scene based on acquired spatial anchor point data information, wherein the current scene is displayed in the head-mounted display device; generating a spatial anchor point corresponding to the spatial anchor point pose information based on the spatial anchor point pose information; in response to determining that an associated scene object included in the spatial anchor point data information exists in the current scene, performing association processing on the spatial anchor point and the associated scene object included in the spatial anchor point data information, so as to share the spatial anchor point included in the spatial anchor point data information.

[0009] Optionally, the method further includes: generating target relative pose information according to the above-mentioned spatial anchor point data information; generating spatial anchor point pose information of the to-be-generated spatial anchor point in the above-mentioned current scene according to the generated target relative pose information.

[0010] Optionally, the method further includes: in response to detecting that the spatial anchor point pose information generation mode is the first anchor point pose information generation mode, generating relative pose information between the scene map information near the anchor point and the map information corresponding to the current scene as target relative pose information according to the scene map information near the anchor point and the map information corresponding to the current scene. The spatial anchor point data information includes the scene map information near the anchor point.

[0011] Optionally, the method further includes: in response to detecting that the spatial anchor point pose information generation method is the second anchor point pose information generation method, and the associated scene object included in the above-mentioned spatial anchor point data information exists in the above-mentioned current scene, according to the above-mentioned anchor point associated scene object information, generating the relative pose information between the associated scene object included in the above-mentioned spatial anchor point data information and the scene object that meets the same preset conditions in the above-mentioned current scene as the target relative pose information, wherein the associated scene object represented by the above-mentioned anchor point associated scene object information is the associated scene object included in the above-mentioned spatial anchor point data information. Wherein, the above-mentioned spatial anchor point data information includes anchor point associated scene object information.

[0012] Optionally, the method also includes: in response to detecting that the spatial anchor point pose information generation method is the above-mentioned first anchor point pose information generation method, determining whether there is a scene object that meets the same preset conditions in the above-mentioned current scene; in response to determining that there is a scene object that meets the above-mentioned preset same conditions in the above-mentioned current scene, determining that the associated scene object included in the above-mentioned spatial anchor point data information exists in the above-mentioned current scene.

[0013] Optionally, the method further includes: in response to determining that there is no scene object satisfying the above-mentioned preset same condition in the above-mentioned current scene, determining that the associated scene object included in the above-mentioned spatial anchor point data information does not exist in the above-mentioned current scene.

[0014] Optionally, the method further includes: attaching a scene object that satisfies the same preset condition in the current scene to the spatial anchor point.

[0015] In a second aspect, some embodiments of the present disclosure provide a head-mounted display device, comprising: one or more processors; a storage device for storing one or more programs; a display screen for imaging in front of the user's eyes; when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of the implementation methods in the first aspect.

[0016] In a third aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any one of the implementations in the first aspect.

[0017] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through a spatial anchor point sharing method of some embodiments of the present disclosure, it is possible to bind the shared spatial anchor point with the corresponding real object in the current scene, so that the fixed posture relationship between the shared spatial anchor point and the corresponding real object in the current scene can be determined, and then the shared spatial anchor point and the corresponding real object can be synchronized to move, thereby further improving the user experience. The reason why there is no fixed posture relationship between the shared spatial anchor point and the corresponding real object in the current scene, and the shared spatial anchor point cannot move synchronously with the corresponding real object, resulting in a poor user experience is that only the position of the shared spatial anchor point in the current scene is determined, and the shared spatial anchor point is not bound to the corresponding real object in the current scene, resulting in no fixed posture relationship between the shared spatial anchor point and the corresponding real object in the current scene, and the shared spatial anchor point cannot move synchronously with the corresponding real object, resulting in a poor user experience. Based on this, the spatial anchor point sharing method of some embodiments of the present disclosure, first, generates the spatial anchor point posture information of the to-be-generated spatial anchor point in the current scene according to the acquired spatial anchor point data information. Among them, the above-mentioned current scene is displayed in the above-mentioned head-mounted display device. Thus, the position and posture of the shared spatial anchor point in the above-mentioned current scene can be obtained. Then, according to the above-mentioned spatial anchor point posture information, a spatial anchor point corresponding to the above-mentioned spatial anchor point posture information can be generated. Thus, a spatial anchor point identical to the above-mentioned shared spatial anchor point can be generated in the current scene. Finally, in response to determining that the associated scene object included in the above-mentioned spatial anchor point data information exists in the above-mentioned current scene, the above-mentioned spatial anchor point and the associated scene object included in the above-mentioned spatial anchor point data information are associated with each other to share the spatial anchor point included in the above-mentioned spatial anchor point data information. Thus, it is possible to bind the shared spatial anchor point to the corresponding real object in the current scene, so that the fixed position and posture relationship between the shared spatial anchor point and the corresponding real object in the current scene can be determined. Thus, even if the map where the spatial anchor point and the corresponding real object are located changes, the fixed position and posture relationship between the spatial anchor point and the corresponding real object will not change. Because it is not just determining the position of the shared spatial anchor point in the current scene, but associating the shared spatial anchor point with the corresponding real object in the current scene, so as to achieve the binding of the shared spatial anchor point with the corresponding real object in the current scene, so as to determine the fixed posture relationship between the shared spatial anchor point and the corresponding real object in the current scene. Therefore, the fixed posture relationship between the shared spatial anchor point and the corresponding real object in the current scene can be determined, and then the shared spatial anchor point and the corresponding real object can be moved synchronously, thereby further improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0019] Figure 1 is an exemplary system architecture diagram of a spatial anchor point sharing method applying some embodiments of the present disclosure;

[0020] Figure 2 is a flow chart of some embodiments of the spatial anchor point sharing method according to the present disclosure;

[0021] Figure 3 is a schematic diagram of an application scenario of the spatial anchor point sharing method disclosed in the present invention;

[0022] Figure 4 are flowcharts of other embodiments of the spatial anchor point sharing method according to the present disclosure;

[0023] Figure 5 It is a schematic diagram of the hardware structure of a head-mounted display device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0024] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0025] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0026] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0027] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0028] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0029] Figure 1 An exemplary system architecture 100 that can be applied to an embodiment of the spatial anchor point sharing method applied to a head mounted display device of the present disclosure is shown.

[0030] like Figure 1 As shown, the exemplary system architecture 100 may include a head mounted display device 11 and a terminal device 12 .

[0031] The head-mounted display device 11 may include one or two display screens 111. The above-mentioned display screens are used to display a virtual interface. In addition, the head-mounted display device 11 also includes a frame 112. In some embodiments, the sensor, processing unit, memory and battery of the head-mounted display device 11 may be placed inside the frame 112. In some optional implementations of some embodiments, one or more components of the sensor, processing unit, memory and battery may also be integrated into another independent accessory (not shown), and connected to the frame 112 via a data cable 13. In some optional implementations of some embodiments, the head-mounted display device 11 may only have a display function and some sensors, and the terminal device 12 provides data processing, data storage, power supply and other capabilities.

[0032] The terminal device 12 may include a touch screen 121 , which may be used to display content. In some embodiments, the head mounted display device 11 and the terminal device 12 may be connected via a data cable 13 .

[0033] It should be understood that Figure 1 The number of head mounted display devices and terminal devices in the embodiment is only for illustration. According to the implementation requirements, any suitable number of head mounted display devices and terminal devices may be provided.

[0034] Further references Figure 2 , shows a process 200 of some embodiments of the spatial anchor point sharing method according to the present disclosure. The spatial anchor point sharing method comprises the following steps:

[0035] Step 201: Generate spatial anchor point position information of the to-be-generated spatial anchor point in the current scene based on the acquired spatial anchor point data information.

[0036] In some embodiments, the execution subject (e.g., head-mounted display device) of the spatial anchor point sharing method may generate the spatial anchor point pose information of the spatial anchor point to be generated in the current scene according to the acquired spatial anchor point data information. The above-mentioned current scene is displayed in the above-mentioned head-mounted display device. The above-mentioned spatial anchor point data information may represent the relevant information of the spatial anchor point to be shared obtained from the target server. The above-mentioned target server may represent the server for storing the relevant information of the spatial anchor point to be shared. For example, the spatial anchor point data information stored by the target server may be shared between different devices, different applications or different users. The above-mentioned spatial anchor point data information may include but is not limited to one of the following: spatial anchor point information, spatial anchor point associated scene object information, and anchor point nearby scene map information. The above-mentioned spatial anchor point information may include the identifier of the spatial anchor point to be shared, the name of the spatial anchor point to be shared, and the original pose of the spatial anchor point to be shared. The identifier of the above-mentioned spatial anchor point to be shared may represent the ID of the spatial anchor point. For example, the ID of the spatial anchor point to be shared may be 01. The original pose of the above-mentioned spatial anchor point to be shared may represent the pose of the spatial anchor point to be shared in the original scene map. The above-mentioned original scene map can represent the SLAM map where the spatial anchor point to be shared is located before sharing. The above-mentioned spatial anchor point associated scene object information can represent the relevant information of the associated scene object. The above-mentioned associated scene object can represent the scene object (real object) associated with the above-mentioned spatial anchor point to be shared in the above-mentioned original scene map. The above-mentioned spatial anchor point associated scene object information may include but is not limited to one of the following: the type of scene object, the pose of the scene object in the above-mentioned original scene map, the feature point of the scene object, and the feature vector of the scene object. Here, the type of the above-mentioned scene object is not limited. For example, the scene object can represent a three-dimensional object or a local point cloud. The above-mentioned anchor point vicinity scene map information can represent a SLAM map centered on the above-mentioned spatial anchor point to be shared and within the target diameter range. Here, the specific value of the above-mentioned target diameter is not limited. The above-mentioned spatial anchor point to be generated can represent the spatial anchor point to be generated in the above-mentioned current scene that is the same as the above-mentioned spatial anchor point to be shared. The above-mentioned spatial anchor point pose information can represent the pose of the spatial anchor point to be generated in the current scene. Here, the map of the above-mentioned current scene is not limited. For example, the map of the current scene can be a SLAM map, a point cloud map scanned by a laser radar, or a model map generated by 3D reconstruction. Figure 3 As shown in (a), the spatial anchor point to be shared represented by the spatial anchor point information may be anchor point B, and the associated scene object information represented by the spatial anchor point associated scene object information may be object C. The scene map represented by the scene map information near the anchor point may be map A.

[0037] In some optional implementations of some embodiments, the execution subject may generate spatial anchor point pose information of the to-be-generated spatial anchor point in the current scene according to the acquired spatial anchor point data information through the following steps:

[0038] The first step is to generate target relative pose information based on the above-mentioned spatial anchor point data information. Among them, the above-mentioned target relative pose information can represent the relative pose between the scene map represented by the scene map information near the above-mentioned anchor point and the map of the above-mentioned current scene, or represent the relative pose between the first scene object and the second scene object. The above-mentioned first scene object can represent the associated scene object represented by the above-mentioned spatial anchor point associated scene object information. The above-mentioned second scene object can represent the same scene object in the above-mentioned current scene as the scene object represented by the above-mentioned spatial anchor point associated scene object information. The above-mentioned relative pose can represent a pose matrix.

[0039] The second step is to generate the spatial anchor point pose information of the to-be-generated spatial anchor point in the current scene according to the generated target relative pose information. In practice, the execution subject may determine the spatial anchor point pose information of the to-be-generated spatial anchor point in the current scene as the product of the original pose of the to-be-shared spatial anchor point included in the spatial anchor point data information and the relative pose.

[0040] In some optional implementations of some embodiments, the execution subject may generate target relative pose information according to the spatial anchor point data information through the following steps:

[0041] The first step, in response to detecting that the spatial anchor point pose information generation method is the first anchor point pose information generation method, generates the relative pose information between the scene map information near the anchor point and the map information corresponding to the current scene as the target relative pose information according to the scene map information near the anchor point and the map information corresponding to the current scene. Among them, the first anchor point pose information generation method can represent the method of generating the target relative pose information by using the visual positioning operation. In practice, the execution subject can use the repositioning method in SLAM or use the point cloud registration algorithm to generate the relative pose information between the scene map information near the anchor point and the map information of the current scene according to the scene map information near the anchor point and the map information corresponding to the current scene.

[0042] Step 202: Generate a spatial anchor point corresponding to the spatial anchor point pose information according to the spatial anchor point pose information.

[0043] In some embodiments, the execution subject may generate a spatial anchor point corresponding to the spatial anchor point pose information according to the spatial anchor point pose information. In practice, the execution subject may use a SLAM algorithm to generate a spatial anchor point corresponding to the spatial anchor point pose information according to the spatial anchor point pose information. As an example, Figure 3 (b) and Figure 3 (c) Figure 3 (b) can characterize scenario 1. Figure 3 (c) can characterize scenario 2. Figure 3 (b1) and Figure 3 (c1) can represent the spatial anchor point B created in scene 1 and scene 2 and the same as the spatial anchor point to be shared. Figure 3 The figure shows that the spatial anchor point to be shared can be shared to scene 1 or scene 2, and can be associated with the corresponding spatial anchor point B in scene 1 or scene 2. Therefore, even if the scene map changes, the fixed posture relationship between the spatial anchor point to be shared and the associated scene object remains unchanged.

[0044] Optionally, after the above step 202, first, in response to detecting that the spatial anchor point pose information generation method is the above first anchor point pose information generation method, the above execution subject may determine whether there is a scene object that meets the same preset conditions in the above current scene. In practice, the above execution subject may determine whether there is a scene object that meets the same preset conditions in the above current scene through the following first verification step:

[0045] In the first step, for each scene object in the current scene, feature information of the scene object can be detected by a feature detection algorithm to obtain each feature information. The feature information may include but is not limited to one of the following: type, posture, feature point, feature vector. The feature detection algorithm may include a posture detection algorithm, a type detection algorithm, a feature point detection algorithm, a feature vector detection algorithm, and a three-dimensional object detection algorithm. The posture detection algorithm may detect the posture of the scene object. The feature point detection algorithm may detect the feature point of the scene object. The type detection algorithm may detect the type of the scene object. The three-dimensional object detection algorithm may detect the type, posture, and feature vector of the scene object. The feature vector detection algorithm may detect the feature vector of the scene object. For example, the type detection algorithm may be a Faster R-CNN algorithm. For example, the posture detection algorithm may be a PnP algorithm. For example, the feature point detection algorithm may be a SIFT algorithm. For example, the feature vector detection algorithm may be a Faster R-CNN algorithm. It should be noted that the execution subject may use the posture detection algorithm included in the feature detection algorithm to detect the posture of the scene object, use the type detection algorithm to detect the type of the scene object, use the feature point detection algorithm to detect the feature points of the scene object, and use the feature vector detection algorithm to detect the feature vector of the scene object, thereby obtaining the feature information of the scene object. Alternatively, the execution subject may directly use the three-dimensional object detection algorithm to detect the type, posture, and feature vector of the scene object, and then use the feature point detection algorithm to detect the feature points of the scene object, thereby obtaining the feature information of the scene object.

[0046] In the second step, in response to determining that there is feature information identical to the anchor-associated scene object information in the above-mentioned various feature information, it is determined that there is a scene object in the above-mentioned current scene that meets the preset identical condition. The above-mentioned preset identical condition may be that there is feature information identical to the anchor-associated scene object information in the above-mentioned various feature information or that the similarity between the scene object in the above-mentioned current scene and the scene object represented by the anchor-associated scene object information is the preset similarity. Here, the specific value of the above-mentioned preset similarity is not limited. For example, the preset similarity may be 100%.

[0047] In a third step, in response to determining that there is no feature information identical to the anchor point associated scene object information in the above-mentioned various feature information, it is determined that there is no scene object satisfying the same preset condition in the above-mentioned current scene.

[0048] Then, in response to determining that a scene object that satisfies the same preset condition exists in the current scene, the execution subject may determine that the scene object included in the spatial anchor point data information exists in the current scene.

[0049] Secondly, in response to determining that there is no scene object satisfying the same preset condition in the current scene, the execution subject may determine that the scene object included in the spatial anchor point data information does not exist in the current scene.

[0050] It should be noted that the method of determining whether there is a scene object that meets the same preset condition in the first verification step can also refer to Figure 4 In the second verification step, it is determined whether there is a scene object that meets the same preset conditions in the current scene.

[0051] Step 203: in response to determining that the associated scene object included in the spatial anchor point data information exists in the current scene, association processing is performed on the spatial anchor point and the associated scene object included in the spatial anchor point data information.

[0052] In some embodiments, in response to determining that the associated scene object included in the spatial anchor point data information exists in the current scene, the execution subject may perform an association process on the spatial anchor point and the associated scene object included in the spatial anchor point data information to share the spatial anchor point included in the spatial anchor point data information. In practice, the execution subject may perform an association process on the spatial anchor point and the scene object in the current scene that is the same as the associated scene object included in the spatial anchor point data information through a SLAM algorithm.

[0053] In some optional implementations of some embodiments, the execution subject may attach scene objects that meet the same preset conditions in the current scene to the spatial anchor point. In practice, the execution subject may attach scene objects that meet the same preset conditions in the current scene to the spatial anchor point through an API interface. The API interface may be an interface for attaching scene objects that meet the same preset conditions in the current scene to the spatial anchor point. For example, the API interface may be an interface provided by ARCore. As an example, by Figure 3 (b2) and Figure 3 As shown in Figure (c2), Figure 3 (b2) and Figure 3 (c2) can represent the scene object C associated with the above-mentioned spatial anchor point B.

[0054] Optionally, after step 203, in response to determining that the associated scene object included in the above-mentioned spatial anchor point data information does not exist in the above-mentioned current scene, it is determined again whether the associated scene object included in the above-mentioned spatial anchor point data information exists in the above-mentioned current scene.

[0055] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through a spatial anchor point sharing method of some embodiments of the present disclosure, it is possible to bind the shared spatial anchor point with the corresponding real object in the current scene, so that the fixed posture relationship between the shared spatial anchor point and the corresponding real object in the current scene can be determined, and then the shared spatial anchor point and the corresponding real object can be synchronized to move, thereby further improving the user experience. The reason why there is no fixed posture relationship between the shared spatial anchor point and the corresponding real object in the current scene, and the shared spatial anchor point cannot move synchronously with the corresponding real object, resulting in a poor user experience is that only the position of the shared spatial anchor point in the current scene is determined, and the shared spatial anchor point is not bound to the corresponding real object in the current scene, resulting in no fixed posture relationship between the shared spatial anchor point and the corresponding real object in the current scene, and the shared spatial anchor point cannot move synchronously with the corresponding real object, resulting in a poor user experience. Based on this, the spatial anchor point sharing method of some embodiments of the present disclosure, first, generates the spatial anchor point posture information of the to-be-generated spatial anchor point in the current scene according to the acquired spatial anchor point data information. Among them, the above-mentioned current scene is displayed in the above-mentioned head-mounted display device. Thus, the position and posture of the shared spatial anchor point in the above-mentioned current scene can be obtained. Then, according to the above-mentioned spatial anchor point posture information, a spatial anchor point corresponding to the above-mentioned spatial anchor point posture information can be generated. Thus, a spatial anchor point identical to the above-mentioned shared spatial anchor point can be generated in the current scene. Finally, in response to determining that the associated scene object included in the above-mentioned spatial anchor point data information exists in the above-mentioned current scene, the above-mentioned spatial anchor point and the associated scene object included in the above-mentioned spatial anchor point data information are associated with each other to share the spatial anchor point included in the above-mentioned spatial anchor point data information. Thus, it is possible to bind the shared spatial anchor point to the corresponding real object in the current scene, so that the fixed position and posture relationship between the shared spatial anchor point and the corresponding real object in the current scene can be determined. Thus, even if the map where the spatial anchor point and the corresponding real object are located changes, the fixed position and posture relationship between the spatial anchor point and the corresponding real object will not change. Because it is not just determining the position of the shared spatial anchor point in the current scene, but associating the shared spatial anchor point with the corresponding real object in the current scene, so as to achieve the binding of the shared spatial anchor point with the corresponding real object in the current scene, so as to determine the fixed posture relationship between the shared spatial anchor point and the corresponding real object in the current scene. Therefore, the fixed posture relationship between the shared spatial anchor point and the corresponding real object in the current scene can be determined, and then the shared spatial anchor point and the corresponding real object can be moved synchronously, thereby further improving the user experience.

[0056] Further references Figure 4, which shows a process 400 of another embodiment of a spatial anchor point sharing method. The process 400 of the spatial anchor point sharing method includes the following steps:

[0057] Step 401, generating target relative pose information according to spatial anchor point data information.

[0058] In some embodiments, the above-mentioned execution entity can generate target relative posture information based on the spatial anchor point data information.

[0059] In some optional implementations of some embodiments, the above-mentioned execution subject may generate target relative pose information according to the spatial anchor point data information through the following steps:

[0060] In the first step, in response to detecting that the spatial anchor point pose information generation mode is the second anchor point pose information generation mode, and the associated scene object included in the above-mentioned spatial anchor point data information exists in the above-mentioned current scene, the relative pose information between the associated scene object included in the above-mentioned spatial anchor point data information and the scene object that meets the same preset conditions in the above-mentioned current scene is generated as the target relative pose information according to the above-mentioned anchor point associated scene object information. Among them, the associated scene object represented by the above-mentioned anchor point associated scene object information is the associated scene object included in the above-mentioned spatial anchor point data information. The above-mentioned second anchor point pose information generation mode can represent the mode of generating the target relative pose information by using image recognition. The above-mentioned spatial anchor point data information can represent the relevant information of the spatial anchor point to be shared obtained from the target server. The above-mentioned target server can represent the server for storing the relevant information of the spatial anchor point to be shared. For example, the spatial anchor point data information stored by the target server can be shared between different devices, different applications or different users. The above-mentioned spatial anchor point data information can include spatial anchor point associated scene object information. The above-mentioned spatial anchor point associated scene object information can represent the relevant information of the associated scene object. The above-mentioned associated scene object can represent the scene object (real object) associated with the above-mentioned spatial anchor point to be shared in the above-mentioned original scene map. The above-mentioned spatial anchor point associated scene object information may include but is not limited to one of the following: the type of scene object, the pose of the scene object in the above-mentioned original scene map, the feature point of the scene object, and the feature vector of the scene object. Here, the type of the above-mentioned scene object is not limited. For example, the scene object can represent a three-dimensional object or a local point cloud. The above-mentioned spatial anchor point data information may include spatial anchor point information. The above-mentioned spatial anchor point information may include the identifier of the spatial anchor point to be shared, the name of the spatial anchor point to be shared, and the original pose of the spatial anchor point to be shared. The identifier of the above-mentioned spatial anchor point to be shared can represent the ID of the spatial anchor point. For example, the ID of the spatial anchor point to be shared can be 01. The original pose of the above-mentioned spatial anchor point to be shared can represent the pose of the spatial anchor point to be shared in the original scene map. The above-mentioned original scene map can represent the SLAM map where the spatial anchor point to be shared is located before sharing. The spatial anchor point to be generated can represent the spatial anchor point that is the same as the spatial anchor point to be shared in the above-mentioned current scene. The spatial anchor point pose information can characterize the pose of the spatial anchor point to be generated in the current scene. Here, there is no limitation on the map of the current scene. For example, the map of the current scene can be a SLAM map, a point cloud map scanned by a lidar, or a model map generated by three-dimensional reconstruction. In practice, first, the execution subject can determine whether there are scene objects that meet the same preset conditions in the current scene. Then, in response to determining that there are scene objects that meet the same preset conditions in the current scene, it is determined that the associated scene objects included in the spatial anchor point data information exist in the current scene.Afterwards, the relocation method in SLAM or the point cloud registration algorithm can be used to generate the relative pose information between the associated scene object included in the above-mentioned spatial anchor point data information and the scene object that meets the same preset conditions in the above-mentioned current scene according to the pose of the scene object in the above-mentioned original scene map included in the above-mentioned anchor point associated scene object information and the pose of the scene object that meets the same preset conditions in the above-mentioned current scene as the target relative pose information. As an example, by. Figure 3 (b1) and Figure 3 As shown in (c1), Figure 3 (b1) and Figure 3 (c1) It can be characterized that the anchor point associated scene object information included in the above-mentioned spatial anchor point data information exists in scene 1 and scene 2.

[0061] In some optional implementations of some embodiments, the execution subject may determine whether there is a scene object that meets the same preset condition in the current scene through the following second verification step:

[0062] The first step is to obtain image information corresponding to the above-mentioned anchor point associated scene object information as target image information. The above-mentioned target image information can represent the image of the above-mentioned anchor point associated scene object information. In practice, the above-mentioned execution subject can obtain the image information corresponding to the above-mentioned anchor point associated scene object information as the target image information from a database storing images of scene objects.

[0063] The second step is to obtain scene image information corresponding to each scene object in the current scene to obtain each scene image information. The scene image information in each scene image information can represent the image of the scene object in the current scene. In practice, the execution subject can obtain the scene image information corresponding to each scene object in the current scene from a database storing images of scene objects to obtain each scene image information.

[0064] The third step is to generate, according to the target image information, each histogram information corresponding to the target image information as the first histogram information group. In practice, the execution subject can use a histogram generation function to generate, according to the target image information, each histogram information corresponding to the target image information as the first histogram information group. For example, the histogram generation function can be the calcHist function under OpenCV.

[0065] The fourth step is to generate, based on the above-mentioned scene image information, each histogram information corresponding to each scene image information in the above-mentioned scene image information as a second histogram information group, and obtain each second histogram information group. Among them, the scene image information in the above-mentioned scene image information corresponds to the second histogram information group in the above-mentioned second histogram information group. In practice, the above-mentioned execution subject can use a histogram generation function to generate, based on the above-mentioned scene image information, each histogram information corresponding to each scene image information in the above-mentioned scene image information as a second histogram information group. For example, the histogram generation function can be the calcHist function under OpenCV.

[0066] The fifth step is to perform image enhancement processing on the target image information and the scene image information according to the first histogram information group and the second histogram information groups to obtain the target image information and the scene image information after image enhancement processing.

[0067] The sixth step is to determine whether there is a scene object that meets the same preset conditions in the current scene based on the target image information and each scene image information after image enhancement processing. In practice, the execution subject can use the cosine similarity method to determine the similarity between the target image information and each scene image information in the scene image information to obtain each similarity information. Then, it is determined whether there is a scene object in the current scene and the similarity corresponding to the preset similarity is used.

[0068] In the process of adopting technical solutions to solve the above-mentioned technical problem one, the following technical problem two is often accompanied: when using image recognition methods to determine whether there are scene objects that meet the same preset conditions in the current scene, the image of the scene object is not pre-processed, resulting in low clarity of the image of the scene object, resulting in low accuracy in confirming whether there are scene objects that meet the same preset conditions based on the image of the scene object with low clarity. For the above-mentioned technical problem two, the conventional solution is generally to use a denoising tool to denoise the image of the scene object. The inventor took into account the shortcomings of using a denoising tool to denoise the image of the scene object. The inventor's unit has experience in solving image quality problems, and combined with the situation of the cooperative unit commissioned for development, we decided to adopt the following solution:

[0069] In some optional implementations of some embodiments, the execution subject may perform image enhancement processing on the target image information and the scene image information according to the first histogram information group and the second histogram information groups through the following steps to obtain the target image information and the scene image information after image enhancement processing:

[0070] In the first step, the first histogram information group and each of the second histogram information groups are determined as a histogram information group set.

[0071] The second step is to determine the pixel mean information of each histogram information in the above histogram information group set. The above pixel mean information can represent the pixel mean of the channel image corresponding to the histogram information. In practice, the above execution subject can use NumPy's np.mean() function to determine the pixel mean corresponding to the above histogram information.

[0072] The third step is to perform the following enhancement steps for each histogram information group in the above histogram information group set:

[0073] The first enhancement step is to determine the histogram information in the histogram information group that meets the preset channel condition as the target histogram information. The preset channel condition may be that the pixel mean of the channel image corresponding to the histogram information in the histogram information group is the largest.

[0074] The second enhancement step is to correct the above-mentioned histogram information group according to the above-mentioned target histogram information to obtain the corrected histogram information. In practice, first, the above-mentioned execution subject can determine each histogram information that is not the above-mentioned target histogram in the above-mentioned histogram information group as each histogram information to be corrected. Then, a histogram specification algorithm can be used to correct the above-mentioned each histogram information to be corrected according to the above-mentioned target histogram to obtain each histogram information after correction.

[0075] In the third enhancement step, the scene image information corresponding to each corrected histogram information and the target histogram information is determined as the first scene image information.

[0076] The fourth enhancement step is to convert the first scene image information to obtain the converted first scene image information. In practice, first, the execution subject may convert the first scene image information to HIS color.

[0077] The fifth enhancement step is to perform equalization processing on the first scene image information after the conversion processing, and obtain the first scene image information after equalization processing as the scene image information after image enhancement processing. In practice, the execution subject can use an image equalization algorithm to perform equalization processing on the first scene image information after the conversion processing through the brightness component of the first scene image information after the conversion processing. For example, the image equalization algorithm can be an adaptive local histogram equalization algorithm.

[0078] As an inventive point of an embodiment of the present disclosure, the above technical solution solves the technical problem that "the accuracy of confirming whether there is a scene object that meets the same preset condition based on the image of the scene object with low definition is low". The factors that lead to the low accuracy of confirming whether there is a scene object that meets the same preset condition based on the image of the scene object with low definition are often as follows: when using the image recognition method to determine whether there is a scene object that meets the same preset condition in the current scene, the image of the scene object is not considered to be preprocessed, resulting in the low definition of the image of the scene object. If the above factors are solved, the effect of improving the accuracy of confirming whether there is a scene object that meets the same preset condition can be achieved. In order to achieve this effect, when determining whether there is a scene object that meets the same preset condition, the present disclosure first performs image enhancement processing on the scene object image to obtain the scene object image after image enhancement processing, and then determines whether there is a scene object that meets the same preset condition in each scene object image after image enhancement processing. Instead of directly confirming based on the image of the scene object, the clarity of the scene object image is improved. Therefore, the image of the scene object with higher definition can be used to confirm whether there is a scene object that meets the same preset condition. As a result, the accuracy of confirming whether there is a scene object that meets the same preset condition based on the image of the scene object with lower definition can be improved.

[0079] It should be noted that the method of determining whether there is a scene object that meets the same preset condition in the above second verification step can also refer to Figure 2 In the first verification step, it is determined whether there is a scene object that meets the same preset conditions in the current scene.

[0080] Step 402: Generate spatial anchor point pose information of the to-be-generated spatial anchor point in the current scene based on the generated target relative pose information.

[0081] In some embodiments, the specific implementation of step 402 and the technical effects brought about can be referred to Figure 2 The second step in some optional implementations of step 201 in the corresponding embodiments will not be described in detail here.

[0082] Step 403: Generate a spatial anchor point corresponding to the spatial anchor point pose information according to the spatial anchor point pose information.

[0083] Step 404, in response to determining that the associated scene object included in the spatial anchor point data information exists in the current scene, the spatial anchor point and the associated scene object included in the spatial anchor point data information are associated to share the spatial anchor point included in the spatial anchor point data information.

[0084] In some embodiments, the specific implementation of step 403 and step 404 and the technical effects brought about can be referred to Figure 2 The corresponding steps 202 and 203 in the embodiments are not described in detail here.

[0085] from Figure 4 It can be seen that Figure 2 Compared with the description of some corresponding embodiments, Figure 4 The process 400 of the spatial anchor point sharing method in some corresponding embodiments embodies a method of using image recognition to confirm whether there are scene objects that meet the same preset conditions in the current scene, and embodies a specific method of generating spatial anchor point posture information based on the target relative posture information, and uses the image of the scene object after enhancement processing for confirmation, thereby improving the image clarity of the scene object, and improving the accuracy of confirming whether there are scene objects that meet the same preset conditions based on the image of the scene object with lower clarity, thereby improving the user experience.

[0086] Reference below Figure 5 , which shows a schematic diagram of the hardware structure of a head-mounted display device 500 with display function.

[0087] like Figure 5 As shown, the head mounted display device 500 includes a processing device (CPU) 501, a memory (ROM) 502, an input unit 503 and an output unit 504, wherein the processing device 501, the memory 502, the input unit 503 and the output unit 504 are connected to each other through a bus 505. Here, the method according to some embodiments of the present disclosure may be implemented as a computer program and stored in the memory 502. The processing device 501 in the head mounted display device 500 specifically implements the spatial anchor point sharing function defined in the method of some embodiments of the present disclosure by calling the above-mentioned computer program stored in the memory 502. In some implementations, the input unit 503 may include devices such as a camera, a microphone, a gyroscope, an accelerometer, a magnetometer, etc., and the output unit 504 may be a device that can be used to display content, such as a display screen. The above-mentioned display screen may be a micro display screen. Therefore, when the processing device 501 calls the above-mentioned computer program to execute the spatial anchor point sharing function, it may control the input unit 503 to obtain the user's gesture, voice and other operation instructions, and control the output unit 504 to display the display content.

[0088] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0089] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0090] The computer-readable medium may be included in the head-mounted display device; or it may exist independently without being assembled into the head-mounted display device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the head-mounted display device, the head-mounted display device: generates spatial anchor point pose information of the to-be-generated spatial anchor point in the current scene according to the acquired spatial anchor point data information, wherein the current scene is displayed in the head-mounted display device; generates a spatial anchor point corresponding to the spatial anchor point pose information according to the spatial anchor point pose information; in response to determining that the associated scene object included in the spatial anchor point data information exists in the current scene, associates the spatial anchor point with the associated scene object included in the spatial anchor point data information to share the spatial anchor point included in the spatial anchor point data information.

[0091] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0094] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A spatial anchor point sharing method, applied to a head mounted display device, comprising: Generate spatial anchor point pose information of the to-be-generated spatial anchor point in the current scene according to the acquired spatial anchor point data information, wherein the current scene is displayed in the head mounted display device; According to the spatial anchor point position information, generating a spatial anchor point corresponding to the spatial anchor point position information; In response to determining that the associated scene object included in the spatial anchor point data information exists in the current scene, the spatial anchor point and the associated scene object included in the spatial anchor point data information are associated with each other to share the spatial anchor point included in the spatial anchor point data information.

2. The method according to claim 1, wherein: The generating, according to the acquired spatial anchor point data information, spatial anchor point pose information of the to-be-generated spatial anchor point in the current scene comprises: Generate target relative position information according to the spatial anchor point data information; According to the generated target relative pose information, the spatial anchor point pose information of the to-be-generated spatial anchor point in the current scene is generated.

3. The method according to claim 2, wherein: The spatial anchor point data information includes scene map information near the anchor point; And generating target relative position information according to the spatial anchor point data information includes: In response to detecting that the spatial anchor point pose information generation method is the first anchor point pose information generation method, based on the scene map information near the anchor point and the map information corresponding to the current scene, the relative pose information between the scene map information near the anchor point and the map information of the current scene is generated as the target relative pose information.

4. The method according to claim 1, wherein: The spatial anchor point data information includes anchor point associated scene object information; and the method further includes: In response to detecting that the spatial anchor point pose information generation method is the second anchor point pose information generation method, and the associated scene object included in the spatial anchor point data information exists in the current scene, based on the anchor point associated scene object information, the relative pose information between the associated scene object included in the spatial anchor point data information and the scene object in the current scene that meets the same preset conditions is generated as the target relative pose information, wherein the associated scene object represented by the anchor point associated scene object information is the associated scene object included in the spatial anchor point data information.

5. The method according to claim 3, wherein: After generating a spatial anchor point corresponding to the spatial anchor point pose information according to the spatial anchor point pose information, the method further includes: In response to detecting that the spatial anchor point pose information generation method is the first anchor point pose information generation method, determining whether there is a scene object that meets the same preset condition in the current scene; In response to determining that a scene object that meets the preset same condition exists in the current scene, it is determined that the associated scene object included in the spatial anchor point data information exists in the current scene.

6. The method according to claim 5, wherein: The method further comprises: In response to determining that there is no scene object satisfying the preset identical condition in the current scene, it is determined that the associated scene object included in the spatial anchor point data information does not exist in the current scene.

7. The method according to claim 1, wherein: The associating processing of the spatial anchor point and the associated scene object included in the spatial anchor point data information includes: The scene objects that exist in the current scene and meet the same preset conditions are attached to the space anchor point.

8. A head mounted display device, comprising: one or more processors; A storage device for storing one or more programs; A display screen for forming an image in front of the user's eyes; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

9. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images