Positioning method, apparatus and electronic device

Abstract

The present disclosure provides a positioning method, device and electronic equipment, relates to the technical field of positioning, and is used for solving the problem of improving the positioning accuracy of the electronic equipment without improving the cost of the electronic equipment. The method comprises the following steps: acquiring a first image collected by each image collection device; determining an actual feature point in the first image; determining image information of the first image according to a historical image collected by the image collection device, a space map constructed according to the historical image and the first image; determining a second pose of the image collection device according to a calibration parameter of the image collection device, a first pose, and position information of the actual feature point; and determining a positioning result of a virtual reality device according to the second pose.

Description

Technical Field

[0001] This disclosure relates to the field of positioning technology, and more particularly to positioning methods, apparatus and electronic devices. Background Technology

[0002] Currently, in positioning scenarios, electronic devices can locate themselves based on specific points in images captured by image acquisition devices. The more feature points there are, the higher the requirements for the image acquisition device, and the higher the positioning accuracy of the electronic device.

[0003] It is evident that maintaining the positioning accuracy of electronic devices comes at the cost of higher costs. Therefore, improving the positioning accuracy of electronic devices without increasing their cost has become a pressing issue. Summary of the Invention

[0004] In view of this, the present disclosure provides positioning methods, apparatus and electronic devices to solve the problem of how to improve the positioning accuracy of electronic devices without increasing the cost of electronic devices.

[0005] To achieve the above objectives, the present disclosure provides the following technical solution:

[0006] In a first aspect, this disclosure provides a positioning method applied to a virtual reality device including multiple image acquisition devices, comprising: acquiring a first image acquired by each image acquisition device; determining actual feature points in the first image; determining image information of the first image based on historical images acquired by the image acquisition devices, a spatial map constructed based on the historical images, and the first image; wherein the image information includes: position information of the actual feature points and / or a first pose; determining a second pose of the image acquisition devices based on calibration parameters of the image acquisition devices, the first pose, and the position information of the actual feature points; and determining the positioning result of the virtual reality device based on the second pose.

[0007] As an optional implementation of this disclosure, the location information includes two-dimensional coordinates and three-dimensional coordinates. Before determining the positioning result of the virtual reality device based on the second pose, the positioning method provided by this disclosure further includes: processing the first image to determine that the transformation relationship between the three-dimensional coordinates and the two-dimensional coordinates is equal to the actual total number of actual feature points of the first pose; if the actual total number is greater than or equal to a preset threshold, optimizing the location information of each historical feature point of the image to be optimized in the spatial map based on the second pose; wherein, the image to be optimized includes historical images corresponding to the first image whose actual total number is greater than or equal to the preset threshold. As an optional implementation of this disclosure, determining the image information of the first image based on historical images acquired by the image acquisition device, a spatial map constructed based on the historical images, and the first image includes: matching historical feature points in the spatial map constructed based on the historical images with actual feature points in the first image to determine the location information corresponding to each actual feature point; and determining the image information of the first image based on the location information corresponding to each actual feature point in the actual image.

[0008] As an optional implementation of this disclosure, the location information includes three-dimensional coordinates and two-dimensional coordinates, with one historical feature point corresponding to one three-dimensional coordinate and one actual feature point corresponding to one two-dimensional coordinate. Matching historical feature points in the spatial map constructed based on historical images with actual feature points in the first image to determine the location information corresponding to each actual feature point includes: matching historical feature points in the spatial map with actual feature points in the first image to determine the historical feature point corresponding to each actual feature point; and performing the following operation on each actual feature point: determining the location information corresponding to the actual feature point based on the two-dimensional coordinates corresponding to the actual feature point and the three-dimensional coordinates of the historical feature points corresponding to the actual feature point.

[0009] As an optional implementation of this disclosure, one historical feature point corresponds to one historical descriptor, and one actual feature point corresponds to one actual descriptor; matching historical feature points in the spatial map with actual feature points in the first image to determine the historical feature point corresponding to each actual feature point includes: matching historical feature points and actual feature points according to the historical descriptors of historical feature points contained in the spatial map and the actual descriptors of actual feature points in the first image to determine the historical feature point corresponding to each actual feature point.

[0010] As an optional implementation of this disclosure, the historical feature points and actual feature points are matched according to the historical descriptors of historical feature points contained in the spatial map and the actual descriptors of actual feature points in the first image to determine the historical feature point corresponding to each actual feature point, including: determining a first distance between the historical descriptors of historical feature points contained in the spatial map and the actual descriptors of actual feature points in the first image; and determining that the historical feature point and the actual feature point corresponding to the first distance correspond when the first distance is less than or equal to a preset threshold.

[0011] As an optional implementation of this disclosure, determining the image information of the first image based on the location information includes: calculating the first image based on the location information corresponding to each actual feature point in the first image to determine the image information of the first image.

[0012] As an optional implementation of this disclosure, when the actual total number is greater than or equal to a preset threshold, before optimizing the position information of each historical feature point of the image to be optimized in the spatial map based on the second pose, the method further includes: matching the historical image and the first image to determine the historical image corresponding to each first image.

[0013] As an optional implementation of this disclosure, one historical image corresponds to one historical global descriptor, and one first image corresponds to one actual global descriptor; matching the historical image and the first image to determine the historical image corresponding to each first image includes: matching the historical image and the first image according to the historical global descriptor of the historical image and the actual global descriptor of the first image to determine the historical image corresponding to each first image.

[0014] As an optional implementation of this disclosure, matching the historical image and the first image based on the historical global descriptor of the historical image and the actual global descriptor of the first image to determine the historical image corresponding to each first image includes: determining a second distance between each historical rights descriptor and each actual global descriptor based on the historical global descriptor of the historical image and the actual global descriptor of the first image; and determining that the historical image corresponding to the second distance and the first image are binary corresponding if the second distance is less than a target threshold.

[0015] Secondly, this disclosure provides a positioning device applied to a virtual reality device including multiple image acquisition devices, comprising: an acquisition unit for acquiring a first image acquired by each image acquisition device; a processing unit for determining actual feature points in the first image acquired by the acquisition unit; the processing unit is further configured to determine image information of the first image based on historical images acquired by the image acquisition devices, a spatial map constructed based on the historical images, and the first image acquired by the acquisition unit; wherein the image information includes: position information of the actual feature points and / or a first pose; the processing unit is further configured to determine a second pose of the image acquisition device based on the calibration parameters of the image acquisition device, the first pose, and the position information of the actual feature points; the processing unit is further configured to determine the positioning result of the virtual reality device based on the second pose.

[0016] As an optional implementation of this disclosure, the location information includes two-dimensional coordinates and three-dimensional coordinates; the processing unit is further configured to solve the first image acquired by the acquisition unit to determine that the transformation relationship between the three-dimensional coordinates and the two-dimensional coordinates is equal to the actual total number of actual feature points of the first pose; the processing unit is further configured to optimize the location information of each historical feature point of the image to be optimized in the spatial map based on the second pose when the actual total number is greater than or equal to a preset threshold; wherein, the image to be optimized includes historical images corresponding to the first image whose actual total number is greater than or equal to the preset threshold.

[0017] As an optional implementation of this disclosure, the processing unit is specifically used to match historical feature points in the spatial map constructed based on historical images with actual feature points in the first image obtained by the acquisition unit, and determine the location information corresponding to each actual feature point; the processing unit is specifically used to determine the image information of the first image based on the location information.

[0018] As an optional implementation of this disclosure, the location information includes three-dimensional coordinates and two-dimensional coordinates, with one historical feature point corresponding to one three-dimensional coordinate and one actual feature point corresponding to one two-dimensional coordinate; the processing unit is specifically used to match the historical feature points in the spatial map with the actual feature points in the first image acquired by the acquisition unit to determine the historical feature point corresponding to each actual feature point; the processing unit is specifically used to perform the following operation on each actual feature point: determine the location information corresponding to the actual feature point based on the two-dimensional coordinates corresponding to the actual feature point and the three-dimensional coordinates of the historical feature points corresponding to the actual feature point.

[0019] As an optional implementation of this disclosure, one historical feature point corresponds to one historical descriptor, and one actual feature point corresponds to one actual descriptor; the processing unit is specifically used to match the historical feature points and actual feature points according to the historical descriptors of the historical feature points contained in the spatial map and the actual descriptors of the actual feature points in the first image obtained by the acquisition unit, and determine the historical feature point corresponding to each actual feature point.

[0020] As an optional implementation of this disclosure, the processing unit is specifically used to determine a first distance between the historical descriptor of the historical feature points contained in the spatial map and the actual descriptor of the actual feature points in the first image acquired by the acquisition unit; the processing unit is specifically used to determine that the historical feature points and actual feature points corresponding to the first distance correspond to each other when the first distance is less than or equal to a preset threshold.

[0021] As an optional implementation of this disclosure, the processing unit is specifically used to solve the first image based on the position information corresponding to each actual feature point in the first image obtained by the acquisition unit, and determine the image information of the first image.

[0022] As an optional implementation of this disclosure, the processing unit is further configured to match the historical images with the first images acquired by the acquisition unit to determine the historical images corresponding to each first image.

[0023] As an optional implementation of this disclosure, one historical image corresponds to one historical global descriptor, and one first image corresponds to one actual global descriptor; the processing unit is specifically used to match the historical image and the first image according to the historical global descriptor of the historical image and the actual global descriptor of the first image obtained by the acquisition unit, and determine the historical image corresponding to each first image.

[0024] As an optional implementation of this disclosure, the processing unit is specifically configured to determine a second distance between each historical rights descriptor and each actual global descriptor based on the historical global descriptor of the historical image and the actual global descriptor of the first image obtained by the acquisition unit; the processing unit is specifically configured to determine that the historical image and the first image corresponding to the second distance are binary corresponding if the second distance is less than a target threshold.

[0025] Thirdly, this disclosure provides an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to cause the electronic device to implement the positioning method provided in the first aspect above when executing the computer program.

[0026] Fourthly, this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a computing device, causes the computing device to implement the positioning method provided in the first aspect above.

[0027] Fifthly, this disclosure provides a computer program product that, when run on a computer, enables the computer to implement the positioning method provided in the first aspect above.

[0028] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on the first computer-readable storage medium. The first computer-readable storage medium may be packaged together with the processor of the positioning device, or it may be packaged separately from the processor of the positioning device; this disclosure does not impose any limitations on this.

[0029] The descriptions of the second, third, fourth, and fifth aspects in this disclosure can be referenced to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth, and fifth aspects can be referenced to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0030] In this disclosure, the names of the aforementioned positioning devices do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of each device or functional module are similar to those of this disclosure, they fall within the scope of the claims of this disclosure and their equivalents.

[0031] These or other aspects of this disclosure will become more readily apparent in the following description.

[0032] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0033] A spatial map is constructed using historical images acquired by an image acquisition device. When locating a virtual reality device, the image information of the first image can be determined based on the historical images, the spatial map, and the first image, ensuring the accuracy of the determined image information. Then, based on the calibration parameters of the image acquisition device, the first pose, and the position information of actual feature points, the second pose of the image acquisition device is determined. Since the first pose and position information are more accurate, the location result of the virtual reality device can be determined more accurately based on the second pose.

[0034] Furthermore, when the electronic device is a virtual reality device, only a spatial map needs to be pre-configured in the electronic device. This does not increase the cost of the electronic device and can also ensure the positioning accuracy of the electronic device, thus solving the problem of how to improve the positioning accuracy of the electronic device without increasing its cost. Attached Figure Description

[0035] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0036] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 A schematic diagram of a scenario for the positioning method provided in this embodiment of the disclosure;

[0038] Figure 2 One of the flowcharts of the positioning method provided in this embodiment of the disclosure;

[0039] Figure 3 A second schematic flowchart illustrating the positioning method provided in this embodiment of the present disclosure;

[0040] Figure 4 The third schematic flowchart of the positioning method provided in the embodiments of this disclosure;

[0041] Figure 5 A fourth schematic flowchart illustrating the positioning method provided in this embodiment of the disclosure;

[0042] Figure 6 Fifth flowchart illustrating the positioning method provided in this embodiment of the disclosure;

[0043] Figure 7 A flowchart illustrating the positioning method provided in this embodiment of the present disclosure is shown in Figure 6.

[0044] Figure 8 Seventh schematic flowchart of the positioning method provided in the embodiments of this disclosure;

[0045] Figure 9 Eighth schematic flowchart of the positioning method provided in the embodiments of this disclosure;

[0046] Figure 10 A flowchart illustrating the positioning method provided in this embodiment of the present disclosure (Figure 9);

[0047] Figure 11 A flowchart illustrating the positioning method provided in this embodiment of the present disclosure is shown in Figure 10.

[0048] Figure 12 A schematic diagram of the positioning device provided in the embodiments of this disclosure;

[0049] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure;

[0050] Figure 14 This is a schematic diagram of the structure of a computer program product for a positioning method provided in an embodiment of this disclosure. Detailed Implementation

[0051] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0052] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0053] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0054] In the embodiments of this disclosure, pose describes the position and orientation of an object (such as a coordinate) in a specified coordinate system.

[0055] In this embodiment of the disclosure, PnP refers to Perspective-n-Poin. PnP is a method for solving point-pair motion from three-dimensional (3D) to two-dimensional (2D).

[0056] COLMAP in this embodiment is a general structure-from-motion (SfM) and multiple view stereo (MVS) pipeline with graphical and command-line interfaces.

[0057] In this embodiment of the disclosure, NetVLAD refers to a local aggregation descriptor vector based on a convolutional neural network.

[0058] Figure 1 This is a schematic diagram of a scene illustrating the positioning method provided in this embodiment, including a virtual reality device 1 with multiple image acquisition devices. After wearing the virtual reality device 1, the user scans the scene using the device to obtain map information of that scene. For example: Figure 1 As shown, a user scans the room using virtual reality device 1, thus storing map information (hereinafter referred to as historical images) of the room in virtual reality device 1. Subsequently, when other users enter the room wearing virtual reality device 1, virtual reality device 1 can capture a first image of the room through an image acquisition device. Then, virtual reality device 1 determines the actual feature points in the first image. Based on the historical images captured by the image acquisition device, the spatial map constructed from the historical images, and the first image, virtual reality device 1 determines the image information of the first image. Then, based on the calibration parameters of the image acquisition device, the first pose, and the position information of the actual feature points, virtual reality device 1 determines the second pose of the image acquisition device. Finally, virtual reality device 1 determines its positioning result based on the second pose. It can be seen that by optimizing the position information of historical feature points in the historical images, the position information of each historical feature point in the historical images becomes more accurate, allowing virtual reality device 1 to more accurately determine the user's current location in the room.

[0059] Specifically, the positioning device provided in this embodiment may be a part of virtual reality device 1, or it may be virtual reality device 1 itself. Virtual reality device 1 may be an electronic device using virtual reality (VR) technology, or an electronic device using augmented reality (AR) technology (such as an AR headset), and there is no limitation here.

[0060] Specifically, the image acquisition device can be a camera, such as a fisheye camera.

[0061] It should be noted that the above example is based on a virtual reality device 1 comprising four image acquisition devices (e.g., image acquisition device 1-1, image acquisition device 1-2, image acquisition device 1-3, and image acquisition device 1-4). In other examples, the virtual reality device 1 may comprise two or more image acquisition devices; this is not a limitation here.

[0062] For example, taking an AR helmet as the execution subject for implementing the positioning method provided in the embodiments of this disclosure, and the AR helmet including multiple image acquisition devices for acquiring environmental images as an example, the positioning method provided in the embodiments of this disclosure will be described.

[0063] Figure 2 This is a flowchart illustrating a positioning method according to an exemplary embodiment, such as... Figure 2 As shown, the method includes the following S1-S5.

[0064] S1. Acquire the first image acquired by each image acquisition device.

[0065] In some examples, when an AR headset includes multiple image acquisition devices, it can simultaneously capture multiple initial images of the current environment. At any given moment, the initial images captured by each device are related. Therefore, by analyzing these multiple initial images captured simultaneously, the AR headset can be positioned. For example, matching the currently captured initial image with historical images in a preset map yields image information for each initial image and the location information of each actual feature point. Then, based on the calibration parameters of the image acquisition devices, the first pose of the initial images captured by each device, and the location information of each actual feature point in the initial images, the second pose of the image acquisition devices is determined. Finally, based on the second pose, the positioning result of the AR headset can be determined.

[0066] Specifically, the camera extrinsic and intrinsic parameters of the image acquisition device, as well as the spatial relationship between the image acquisition device and the AR helmet, are all pre-configured.

[0067] S2. Determine the actual feature points in the first image.

[0068] In some examples, a target extraction algorithm is used to extract feature points from the first image to determine the actual feature points in the first image. For example, the target extraction algorithm can be any of the following: Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), optical flow, or block matching.

[0069] S3. Based on historical images acquired by the image acquisition device, a spatial map constructed from the historical images, and the first image, determine the image information of the first image. The image information includes: the location information of the actual feature points and / or the first pose.

[0070] In some examples, the positioning method provided in this disclosure pre-captures images of the user's scene, thereby using pre-determined historical images of that scene as prior information. Then, visual positioning is performed based on this prior information. Simultaneously, the spatial map constructed from the historical images can be updated during positioning, allowing the AR headset to more accurately determine its current location and improving the robustness of the positioning. Specifically, the spatial map construction process is as follows:

[0071] 1. The AR headset acquires images of the scene requiring spatial mapping (e.g., a room), obtaining historical images of the room from various angles. Then, feature points are extracted from all historical images, such as scale-invariant feature transform (SIFT) points or deep learning-based feature points, resulting in at least one historical feature point and a historical descriptor for each. Subsequently, a maximum suppression algorithm is used to process the extracted historical feature points, yielding optimized historical feature points.

[0072] 2. Process all historical images captured by the AR helmet to identify similar images. For example, calculate the historical global descriptor for each historical image using NetVLAD. Then, calculate the distance between the historical global descriptors of two historical images, and determine similar images based on the distance. For example, if the distance between historical image 1 and historical image 2 is less than a distance threshold, then historical image 1 and historical image 2 are determined to be similar images.

[0073] It should be noted that the above example illustrates how similar images are determined by calculating the distance between the historical global descriptors of two historical images and then using that distance. In some examples, similar images can also be determined based on the capture time of historical communications. For instance, if the capture time between historical image 1 and historical image 2 is less than a time threshold, then historical image 1 and historical image 2 are considered similar images. In other examples, historical images captured simultaneously by the AR headset's image acquisition devices are considered similar images. For example, if the AR headset contains two image acquisition devices, image acquisition device 1 and image acquisition device 2, then historical image 1 captured by image acquisition device 1 and historical image 2 captured by image acquisition device 2 are considered similar images because they were captured at the same time. Ultimately, users can choose the method for determining similar images based on their actual needs; no specific limitations are imposed here.

[0074] 3. Use colmap to incrementally build maps of similar images frame by frame to obtain the spatial map to be optimized.

[0075] 4. An optimization factor map is used to optimize historical images in the spatial map to be optimized, resulting in an optimized spatial map. The optimized spatial map is then stored in the AR headset's memory. The optimized spatial map includes the 3D coordinates of each historical feature point, a historical descriptor, and a historical global descriptor for each historical image.

[0076] As can be seen, by capturing images of the user's surroundings, the AR headset, pre-stores a spatial map of that scene. Thus, the AR headset can determine its location based on historical images, the spatial map, and the currently captured first image.

[0077] Specifically, the method for extracting the actual feature points contained in the first image and the actual descriptor corresponding to each actual feature point is the same as the method for extracting the historical feature points and the historical descriptor corresponding to each historical feature point in the historical communication, and will not be repeated here.

[0078] Specifically, the first image and the historical images were captured in the same physical space, such as the same room or the same corridor.

[0079] S4. Based on the calibration parameters of the image acquisition device, the first pose, and the position information of the actual feature points, determine the second pose of the image acquisition device.

[0080] In some examples, the location information includes 3D coordinates and 2D coordinates. The 3D coordinates are the 3D coordinates of historical feature points that match the actual feature points, and the 2D coordinates are the pixel coordinates of the actual feature points in the first image. Before determining the second pose, an optimization function needs to be constructed, such as:

[0081] Among them, K i T represents the camera intrinsic parameters of the i-th camera. i0 T represents the relative extrinsic parameter of the i-th camera relative to the 0th camera. 0w P represents the first pose of the first image captured by the 0th camera. mwi p represents the 3D coordinates of the m-th actual feature point in the first image captured by the i-th camera. mwi Let represent the two-dimensional coordinates of the m-th actual feature point in the first image captured by the i-th camera, N represent the total number of cameras in the AR helmet, i∈[0,N], and both i and N are integers greater than or equal to 0, M represent the total number of actual feature points in the first image captured by the i-th camera, m∈[0,M], and both i and M are integers greater than or equal to 0, the 0th camera is any one of the cameras in the AR helmet, and the i-th camera is any camera in the AR helmet except the 0th camera.

[0082] Ideally, cost equals 0, but in practical applications, cost is usually greater than 0. To optimize the obtained second pose, the first pose of the first image captured by the 0th camera needs to be used as the initial value for optimization. Then, the poses related to the first pose are calculated using a target algorithm and substituted into the optimization function. For example, the target algorithm calculates 10 poses related to the first pose, and the first pose and these 10 poses are substituted into the optimization function to determine the cost value. Finally, the pose corresponding to the minimum cost value is determined as the second pose of the image acquisition device.

[0083] S5. Determine the positioning result of the virtual reality device based on the second pose.

[0084] In some examples, the spatial relationship between the virtual reality device and the image acquisition device is predetermined. Therefore, after determining the second pose of the image acquisition device, the pose of the virtual reality device can be determined based on the predetermined spatial relationship between the image acquisition device and the virtual reality device, as well as the second pose. Thus, the positioning result of the virtual reality device can be determined based on its pose.

[0085] Specifically, a virtual reality device can be an entity that performs the positioning method provided in the embodiments of this disclosure, such as an AR headset.

[0086] As described above, the positioning method provided in this embodiment of the present disclosure pre-configures a spatial map in the VR headset. When positioning the VR headset is required, the image information of the first image can be determined based on the first image captured by the image acquisition device and historical images, using the spatial map constructed from the historical images. This ensures the accuracy of the image information. Subsequently, the second pose of the image acquisition device is determined based on the calibration parameters of the image acquisition device, the first pose, and the position information of the actual feature points. Since the first pose and position information are more accurate, determining the positioning result of the VR headset based on the second pose allows for a more accurate determination of the virtual reality device's positioning.

[0087] As an optional implementation of this disclosure, the location information includes two-dimensional coordinates and three-dimensional coordinates; combined with Figure 2 ,like Figure 3 As shown, the positioning method provided in this embodiment of the present disclosure further includes S6 and S7.

[0088] S6. Solve the first image to determine the transformation relationship between the three-dimensional coordinates and the two-dimensional coordinates, which is equal to the actual total number of actual feature points of the first pose.

[0089] In some examples, when solving the first image, the first image can be solved based on the position information corresponding to each actual feature point in the first image, and the transformation relationship between the three-dimensional coordinates and the two-dimensional coordinates can be determined to be equal to the actual total number of actual feature points of the first pose.

[0090] In some examples, when solving the first image based on the position information corresponding to each actual feature point in the first image, PnP solution can be performed on the first image based on the position information corresponding to each actual feature point in the first image. In this way, it can be determined that the transformation relationship between the three-dimensional coordinates and the two-dimensional coordinates is equal to the actual total number of actual feature points of the first pose.

[0091] Specifically, when performing PnP computation on the first image, the Epnp algorithm, the Upnp algorithm, or the P3P algorithm can be used to compute the first image; no specific method is used here.

[0092] S7. When the actual total number is greater than or equal to a preset threshold, optimize the position information of each historical feature point of the image to be optimized in the spatial map based on the second pose; wherein, the image to be optimized includes the historical image corresponding to the first image whose actual total number is greater than or equal to the preset threshold.

[0093] In some examples, the positioning accuracy of the AR headset can be improved by selecting images with sufficient observations from the first image and updating historical feature points in the preset map. For instance, when the actual total number is greater than or equal to a preset threshold, the first image is considered to have sufficient observations. Then, using a second pose, the positional information (such as 3D coordinates) of historical feature points in the historical images corresponding to the first image is optimized, resulting in more accurate 3D coordinates. Because the 3D coordinates of historical map points in the preset map are more accurate, the positioning accuracy of the AR headset can be improved when using the preset map for positioning.

[0094] As an optional implementation of this disclosure, combined with Figure 2 ,like Figure 4 As shown, the above S3 can be implemented through the following S30 and S31.

[0095] S30. Match the historical feature points in the spatial map constructed based on the historical images with the actual feature points in the first image to determine the spatial map of the location information corresponding to each actual feature point.

[0096] Specifically, when locating an AR headset, it's necessary to match historical feature points with actual feature points to find the historical feature points corresponding to each actual feature point. This allows us to determine the location information of each actual feature point, such as its pixel coordinates in the first image and its corresponding 3D coordinates. Thus, based on the location information of each actual feature point in the first image, the image information of the first image can be determined.

[0097] S31. Determine the image information of the first image based on the location information.

[0098] As an optional implementation of this disclosure, the location information includes three-dimensional coordinates and two-dimensional coordinates, with one historical feature point corresponding to one three-dimensional coordinate and one actual feature point corresponding to one two-dimensional coordinate; combined with Figure 4 ,like Figure 5 As shown, the above S30 can be implemented by the following S300 and S301.

[0099] S300. Match the historical feature points in the spatial map with the actual feature points in the first image to determine the spatial map of historical feature points corresponding to each actual feature point.

[0100] In some examples, historical feature points in the spatial map are matched with actual feature points in the first image. For instance, k-nearest neighbor classification is used to match historical feature points in the spatial map with actual feature points in the first image to determine the historical feature point corresponding to each actual feature point. Alternatively, a random sampling consensus algorithm is used to match historical feature points in the spatial map with actual feature points in the first image to determine the historical feature point corresponding to each actual feature point. Or, the historical feature point corresponding to each actual feature point is determined based on the distance between the historical descriptor of the historical feature point and the actual descriptor of the actual feature point.

[0101] S301. Perform the following operation for each actual feature point: Determine the location information corresponding to the actual feature point based on the two-dimensional coordinates of the actual feature point and the three-dimensional coordinates of the historical feature points corresponding to the actual feature point.

[0102] In some examples, when matching a first image with historical images, an actual feature point in the first image corresponds to a historical feature point in the historical image, since each historical feature point corresponds to a three-dimensional coordinate. Therefore, the three-dimensional coordinates of the historical feature point corresponding to the actual feature point can be used as the three-dimensional coordinates of the actual feature point. Since each actual feature point corresponds to a two-dimensional coordinate (i.e., the pixel coordinates of the actual feature point in the first image), a correspondence can be established between the two-dimensional coordinates of the actual feature point and the three-dimensional coordinates of the historical feature point, thus obtaining the positional information corresponding to the actual feature point. Subsequently, based on the positional information corresponding to each actual feature point in the first image, the image information of the first image can be determined.

[0103] As an optional implementation of this disclosure, one historical feature point corresponds to one historical descriptor, and one actual feature point corresponds to one actual descriptor; combined with Figure 5 ,like Figure 6 As shown, the above S300 can be implemented by the following S3000.

[0104] S3000: Based on the historical descriptors of historical feature points contained in the spatial map and the actual descriptors of actual feature points in the first image, match the historical feature points and actual feature points to determine the historical feature points corresponding to each actual feature point.

[0105] In some examples, when matching historical feature points and actual feature points, the historical feature point corresponding to each actual feature point can be determined based on a first distance (such as Euclidean distance) between the historical descriptor of the historical feature point and the actual descriptor of the actual feature point. For example, if the first distance is less than or equal to a preset threshold, the historical feature point and the actual feature point corresponding to the first distance are determined to be consistent.

[0106] In other examples, when matching historical feature points and actual feature points, the historical feature point corresponding to each actual feature point can be determined based on the cosine similarity between the historical descriptor of the historical feature point and the actual descriptor of the actual feature point. For example, if the cosine similarity is less than or equal to the cosine threshold, the historical feature point and the actual feature point corresponding to the first distance are determined to be compatible.

[0107] As an optional implementation of this disclosure, combined with Figure 6 ,like Figure 7 As shown, the above S3000 can be specifically implemented through the following S30000 and S30001.

[0108] S30000: Determine the first distance between the historical descriptors of historical feature points contained in the spatial map and the actual descriptors of actual feature points in the first image.

[0109] S30001. If the first distance is less than or equal to a preset threshold, determine that the historical feature point and the actual feature point corresponding to the first distance are corresponding.

[0110] As an optional implementation of this disclosure, combined with Figure 4 ,like Figure 8 As shown, the above S31 can be implemented by the following S310.

[0111] S310. Based on the position information corresponding to each actual feature point in the first image, the first image is solved to determine the image information of the first image.

[0112] In some examples, when solving the first image based on the position information corresponding to each actual feature point in the first image, PnP solution can be performed on the first image based on the position information corresponding to each actual feature point in the first image, so as to determine the first pose and the actual total number of the first image.

[0113] Specifically, when performing PnP computation on the first image, the Epnp algorithm, the Upnp algorithm, or the P3P algorithm can be used to compute the first image; no specific method is used here.

[0114] As an optional implementation of this disclosure, combined with Figure 3 ,like Figure 9 As shown, the positioning method provided in this embodiment of the present disclosure further includes: S8.

[0115] S8. Match the historical images with the first image to determine the historical image corresponding to each first image.

[0116] In some examples, determining the historical image corresponding to each first image means determining the historical image that is similar to the first image.

[0117] Specifically, the process of determining the historical image corresponding to each first image is similar to the process of determining similar images in the historical images, and will not be elaborated here.

[0118] As an optional implementation of this disclosure, one historical image corresponds to one historical global descriptor, and one first image corresponds to one actual global descriptor; combined with Figure 9 ,like Figure 10 As shown, the above S8 can be implemented through the following S80.

[0119] S8. Match the historical images and the first images based on the historical global descriptors of the historical images and the actual global descriptors of the first images to determine the historical images corresponding to each first image.

[0120] Specifically, the process of determining the actual global descriptor of the first image is the same as the process of determining the historical global descriptor of historical images, and will not be repeated here.

[0121] As an optional implementation of this disclosure, combined with Figure 10 ,like Figure 11 As shown, the above S80 can be implemented by the following S800 and S801.

[0122] S800. Based on the historical global descriptor of the historical image and the actual global descriptor of the first image, determine the second distance between each historical rights descriptor and each actual global descriptor.

[0123] In some examples, the Euclidean distance between the historical global descriptor and the actual global descriptor can be calculated and used as a second distance between them.

[0124] S801. If the second distance is less than the target threshold, determine that the historical image corresponding to the second distance corresponds to the binary representation of the first image.

[0125] The foregoing primarily describes the solutions provided by the embodiments of the present invention from a methodological perspective. To achieve the aforementioned functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0126] In this embodiment of the invention, the positioning device can be divided into functional modules according to the above method example. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0127] like Figure 12 As shown, an embodiment of the present invention provides a schematic diagram of the structure of a positioning device 10. The positioning device 10 includes an acquisition unit 101 and a processing unit 102.

[0128] The acquisition unit 101 is used to acquire a first image acquired by each image acquisition device; the processing unit 102 is used to determine the actual feature points in the first image acquired by the acquisition unit 101; the processing unit 102 is also used to determine the image information of the first image based on the historical images acquired by the image acquisition devices, the spatial map constructed based on the historical images, and the first image acquired by the acquisition unit 101; wherein, the image information includes: the position information of the actual feature points and / or the first pose; the processing unit 102 is also used to determine the second pose of the image acquisition device based on the calibration parameters of the image acquisition device, the first pose, and the position information of the actual feature points; the processing unit 102 is also used to determine the positioning result of the virtual reality device based on the second pose.

[0129] As an optional implementation of this disclosure, the location information includes two-dimensional coordinates and three-dimensional coordinates; the processing unit 102 is further configured to perform calculations on the first image acquired by the acquisition unit 101 to determine that the transformation relationship between the three-dimensional coordinates and the two-dimensional coordinates is equal to the actual total number of actual feature points of the first pose; the processing unit 102 is further configured to optimize the location information of each historical feature point of the image to be optimized in the spatial map based on the second pose when the actual total number is greater than or equal to a preset threshold; wherein, the image to be optimized includes historical images corresponding to the first image whose actual total number is greater than or equal to the preset threshold.

[0130] As an optional implementation of this disclosure, the processing unit 102 is specifically used to match the historical feature points in the spatial map constructed based on the historical images with the actual feature points in the first image acquired by the acquisition unit 101, and determine the location information corresponding to each actual feature point; the processing unit 102 is specifically used to determine the image information of the first image based on the location information.

[0131] As an optional implementation of this disclosure, the location information includes three-dimensional coordinates and two-dimensional coordinates, with one historical feature point corresponding to one three-dimensional coordinate and one actual feature point corresponding to one two-dimensional coordinate; the processing unit 102 is specifically used to match the historical feature points in the spatial map with the actual feature points in the first image acquired by the acquisition unit 101 to determine the historical feature point corresponding to each actual feature point; the processing unit 102 is specifically used to perform the following operation on each actual feature point: determine the location information corresponding to the actual feature point based on the two-dimensional coordinates corresponding to the actual feature point and the three-dimensional coordinates of the historical feature points corresponding to the actual feature point.

[0132] As an optional implementation of this disclosure, one historical feature point corresponds to one historical descriptor, and one actual feature point corresponds to one actual descriptor; the processing unit 102 is specifically used to match the historical feature points and actual feature points according to the historical descriptors of the historical feature points contained in the spatial map and the actual descriptors of the actual feature points in the first image obtained by the acquisition unit 101, and determine the historical feature point corresponding to each actual feature point.

[0133] As an optional implementation of this disclosure, the processing unit 102 is specifically used to determine a first distance between the historical descriptor of the historical feature points contained in the spatial map and the actual descriptor of the actual feature points in the first image acquired by the acquisition unit 101; the processing unit 102 is specifically used to determine that the historical feature points and actual feature points corresponding to the first distance correspond to each other when the first distance is less than or equal to a preset threshold.

[0134] As an optional implementation of this disclosure, the processing unit 102 is specifically used to solve the first image based on the position information corresponding to each actual feature point in the first image obtained by the acquisition unit 101, and determine the image information of the first image.

[0135] As an optional implementation of this disclosure, the processing unit 102 is further configured to match the historical images with the first images acquired by the acquisition unit 101 to determine the historical images corresponding to each first image.

[0136] As an optional implementation of this disclosure, one historical image corresponds to one historical global descriptor, and one first image corresponds to one actual global descriptor; the processing unit 102 is specifically used to match the historical image and the first image according to the historical global descriptor of the historical image and the actual global descriptor of the first image obtained by the acquisition unit 101, and determine the historical image corresponding to each first image.

[0137] As an optional implementation of this disclosure, the processing unit 102 is specifically used to determine a second distance between each historical rights descriptor and each actual global descriptor based on the historical global descriptor of the historical image and the actual global descriptor of the first image obtained by the acquisition unit 101; the processing unit 102 is specifically used to determine that the historical image corresponding to the second distance and the first image are binary corresponding when the second distance is less than a target threshold.

[0138] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and their functions will not be repeated here.

[0139] Of course, the positioning device 10 provided in this embodiment of the invention includes, but is not limited to, the modules described above. For example, the positioning device 10 may also include a storage unit 103. The storage unit 103 may be used to store the program code of the positioning device 10, and may also be used to store data generated by the positioning device 10 during operation, such as data in write requests.

[0140] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 13 As shown, the electronic device may include at least one processor 51, a memory 52, a communication interface 53, and a communication bus 54.

[0141] The following is combined with Figure 13 A detailed introduction to each component of the electronic device:

[0142] The processor 51 is the control center of the electronic device. It can be a single processor or a collective term for multiple processing elements. For example, the processor 51 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more DSPs, or one or more field-programmable gate arrays (FPGAs).

[0143] In a specific implementation, as one example, processor 51 may include one or more CPUs, for example... Figure 13 CPU0 and CPU1 are shown in the diagram. Furthermore, as one embodiment, the electronic device 10 may include multiple processors, such as... Figure 13 The processors 51 and 56 shown are illustrated. Each of these processors can be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0144] The memory 52 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or it may be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory 52 may exist independently and be connected to the processor 51 via a communication bus 54. The memory 52 may also be integrated with the processor 51.

[0145] In a specific implementation, memory 52 is used to store data from this invention and the software program for executing this invention. Processor 51 can perform various functions of the air conditioner by running or executing the software program stored in memory 52 and by calling the data stored in memory 52.

[0146] Communication interface 53 uses any transceiver-like device for communicating with other devices or communication networks, such as Radio Access Network (RAN), Wireless Local Area Networks (WLAN), terminals, and the cloud. Communication interface 53 may include acquisition unit 101 to implement acquisition functions.

[0147] The communication bus 54 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 13 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0148] As an example, combined Figure 12The functions implemented by the acquisition unit 101 in the positioning device 10 are the same as those of the acquisition unit 101 in the positioning device 10. Figure 13 The communication interface 53 has the same function, and the processing unit 102 of the positioning device 10 performs the same function. Figure 13 The processor 51 in the positioning device 10 has the same function as the storage unit 103 in the positioning device 10. Figure 13 The memory 52 in it has the same function.

[0149] Another embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a computing device, causes the computing device to implement the method shown in the above-described method embodiment.

[0150] In some embodiments, the disclosed method may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art.

[0151] Figure 14 A conceptual partial view of a computer program product provided in an embodiment of the present invention is shown schematically. The computer program product includes a computer program for executing computer processes on a computing device.

[0152] In one embodiment, the computer program product is provided using signal bearer medium 410. Signal bearer medium 410 may include one or more program instructions that, when executed by one or more processors, can provide the above-mentioned... Figure 2 The described function or part of the function. Therefore, for example, refer to... Figure 2 In the embodiment shown, one or more features of S1-S5 can be provided by one or more instructions associated with the signal carrying medium 410. Furthermore, Figure 14 The program instructions in the document also describe example instructions.

[0153] In some examples, the signal carrying medium 410 may include a computer-readable medium 411, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a digital magnetic tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc.

[0154] In some implementations, the signal carrying medium 410 may include a computer recordable medium 412, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, and so on.

[0155] In some implementations, the signal carrying medium 410 may include a communication medium 413, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0156] The signal-bearing medium 410 can be transmitted by a wireless communication medium 413 (e.g., a wireless communication medium conforming to the IEEE 802.41 standard or other transmission protocols). One or more program instructions can be, for example, computer-executable instructions or logical implementation instructions.

[0157] In some examples, such as targeting Figure 12 The described positioning device 10 can be configured to provide various operations, functions, or actions in response to one or more program instructions in a computer-readable medium 411, a computer-recordable medium 412, and / or a communication medium 413.

[0158] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0159] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0160] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0161] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0162] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0163] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A positioning method, characterized in that, Virtual reality devices that include multiple image acquisition devices include: Acquire the first image captured by each image acquisition device; Determine the actual feature points in the first image; Based on historical images acquired by the image acquisition device, a spatial map constructed from the historical images, and a first image of the spatial map, image information of the first image is determined; wherein, the image information includes: the location information and / or first pose of the actual feature points; Based on the calibration parameters of the image acquisition device, the first pose, and the position information of the actual feature points, the second pose of the image acquisition device is determined. The positioning result of the virtual reality device is determined based on the second pose; The step of determining the image information of the first image based on historical images acquired by the image acquisition device, a spatial map constructed based on the historical images, and the first image includes: The historical feature points in the spatial map constructed based on the historical images are matched with the actual feature points in the first image to determine the location information corresponding to each actual feature point; Based on the location information, the image information of the first image is determined.

2. The positioning method according to claim 1, characterized in that, The location information includes two-dimensional coordinates and three-dimensional coordinates; Before determining the positioning result of the virtual reality device based on the second pose, the method further includes: The first image is solved to determine that the transformation relationship between three-dimensional coordinates and two-dimensional coordinates is equal to the actual total number of actual feature points of the first pose. If the actual total number is greater than or equal to a preset threshold, the position information of each historical feature point of the image to be optimized in the spatial map is optimized based on the second pose; wherein, the image to be optimized includes the historical image corresponding to the first image whose actual total number is greater than or equal to the preset threshold.

3. The positioning method according to claim 1, characterized in that, The location information includes three-dimensional coordinates and two-dimensional coordinates. One historical feature point corresponds to one three-dimensional coordinate, and one actual feature point corresponds to one two-dimensional coordinate. The step of matching historical feature points in the spatial map constructed based on the historical images with actual feature points in the first image to determine the location information spatial map corresponding to each actual feature point includes: The historical feature points in the spatial map are matched with the actual feature points in the first image to determine the historical feature points corresponding to each actual feature point; For each actual feature point, the following operation is performed: based on the two-dimensional coordinates corresponding to the actual feature point and the three-dimensional coordinates of the historical feature points corresponding to the actual feature point, the location information corresponding to the actual feature point is determined.

4. The positioning method according to claim 3, characterized in that, One historical feature point corresponds to one historical descriptor, and one actual feature point corresponds to one actual descriptor; The step of matching historical feature points in the spatial map with actual feature points in the first image to determine the historical feature point spatial map corresponding to each actual feature point includes: Based on the historical descriptors of historical feature points contained in the spatial map and the actual descriptors of actual feature points in the first image, the historical feature points and the actual feature points are matched to determine the historical feature point corresponding to each actual feature point.

5. The positioning method according to claim 4, characterized in that, The step of matching the historical feature points and the actual feature points based on the historical descriptors of the historical feature points contained in the spatial map and the actual feature points in the first image to determine the historical feature point corresponding to each actual feature point includes: Determine a first distance between the historical descriptors of historical feature points contained in the spatial map and the actual descriptors of actual feature points in the first image; If the first distance is less than or equal to a preset threshold, the historical feature point and the actual feature point corresponding to the first distance are determined to be corresponding.

6. The positioning method according to claim 1, characterized in that, Determining the image information of the first image based on the location information includes: Based on the position information corresponding to each actual feature point in the first image, the first image is solved to determine the image information of the first image.

7. The positioning method according to claim 2, characterized in that, Before optimizing the position information of each historical feature point of the image to be optimized in the spatial map based on the second pose when the actual total number is greater than or equal to a preset threshold, the method further includes: The historical images and the first image are matched to determine the historical image corresponding to each first image.

8. The positioning method according to claim 7, characterized in that, One historical image corresponds to one historical global descriptor, and one first image corresponds to one actual global descriptor; The step of matching the historical images with the first images to determine the historical images corresponding to each first image includes: Based on the historical global descriptor of the historical image and the actual global descriptor of the first image, the historical image and the first image are matched to determine the historical image corresponding to each first image.

9. The positioning method according to claim 8, characterized in that, The step of matching the historical images and the first images based on the historical global descriptors of the historical images and the actual global descriptors of the first images to determine the historical images corresponding to each first image includes: Based on the historical global descriptor of the historical image and the actual global descriptor of the first image, a second distance is determined between each historical global descriptor and each actual global descriptor; If the second distance is less than the target threshold, the historical image corresponding to the second distance is determined to be binary corresponding to the first image.

10. A positioning device, characterized in that, Virtual reality devices that include multiple image acquisition devices include: An acquisition unit is used to acquire the first image acquired by each image acquisition device; The processing unit is used to determine the actual feature points in the first image acquired by the acquisition unit; The processing unit is further configured to determine image information of the first image based on historical images acquired by the image acquisition device, a spatial map constructed based on the historical images, and the first image acquired by the acquisition unit; wherein the image information includes: the location information of the actual feature points and / or the first pose; The processing unit is further configured to determine the second pose of the image acquisition device based on the calibration parameters of the image acquisition device, the first pose, and the position information of the actual feature points. The processing unit is further configured to determine the positioning result of the virtual reality device based on the second pose; the step of determining the image information of the first image based on the historical images acquired by the image acquisition device, the spatial map constructed based on the historical images, and the first image includes: The historical feature points in the spatial map constructed based on the historical images are matched with the actual feature points in the first image to determine the location information corresponding to each actual feature point; Based on the location information, the image information of the first image is determined.

11. An electronic device, characterized in that, include: A memory and a processor, the memory being used to store a computer program; the processor being used to cause the electronic device to implement the positioning method according to any one of claims 1-9 when executing the computer program.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a computing device, causes the computing device to implement the positioning method according to any one of claims 1-9.

13. A computer program product, characterized in that, When the computer program product is run on a computer, the computer enables the positioning method as described in any one of claims 1-9.

Images