Image positioning method and device, electronic equipment, storage medium and program product

Abstract

The application provides a positioning method and device of an image, an electronic device, a storage medium and a program product. The method comprises the following steps: performing regional division on a panoramic image to obtain a first region and a second region; performing feature extraction on the first region to obtain a first feature point matching relationship, and performing feature extraction on the second region to obtain a second feature point matching relationship; determining a first weight based on the first feature point matching relationship, and determining a second weight based on the second feature point matching relationship; and performing weighted calculation on a sampling probability of the first region and a sampling probability of the second region according to the first weight and the second weight to obtain a positioning result. According to the application, the panoramic image is divided into different ground regions and background regions, the weights of the sampling probabilities of different regions are determined after feature extraction on the ground regions and the background regions, and the positioning result is obtained after weighted calculation, so that the positioning effect of the panoramic image is improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to an image localization method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] With the development of panoramic camera technology, it is being applied in more and more fields. Currently, the Simultaneous Localization and Mapping (SLAM) scheme for panoramic cameras mainly uses visual feature matching and tracking to estimate pose and build maps in real time. However, this method suffers from errors in feature matching between neighboring frames, and these errors tend to accumulate as the SLAM system runs. Ultimately, even if loop closure is detected, convergence cannot be optimized well, resulting in poor image localization performance. Summary of the Invention

[0003] This application provides an image positioning method, apparatus, electronic device, storage medium, and program product to solve the problem of poor image positioning performance in the prior art.

[0004] To solve the above problems, this application is implemented as follows:

[0005] In a first aspect, embodiments of this application provide an image localization method, the method comprising:

[0006] The panoramic image is divided into regions to obtain a first region and a second region. The first region is the ground region in the panoramic image, and the second region is the background region in the panoramic image.

[0007] Feature extraction is performed on the first region to obtain the first feature point matching relationship, and feature extraction is performed on the second region to obtain the second feature point matching relationship;

[0008] A first weight is determined based on the first feature point matching relationship, and a second weight is determined based on the second feature point matching relationship. The first weight is the weight value of the sampling probability of the first region, and the second weight is the weight value of the sampling probability of the second region.

[0009] The sampling probabilities of the first region and the second region are weighted and calculated based on the first weight and the second weight to obtain the positioning result.

[0010] Optionally, the step of extracting features from the first region to obtain the first feature point matching relationship, and extracting features from the second region to obtain the second feature point matching relationship, includes:

[0011] The first feature tracking algorithm is used to perform image matching on the first feature points included in the first region to obtain the matching relationship of the first feature points;

[0012] The second feature tracking algorithm is used to perform feature matching and feature tracking on the second feature points included in the second region to obtain the matching relationship of the second feature points.

[0013] Optionally, the first feature point matching relationship includes the number of matching first feature points included in the first region, and the second feature point matching relationship includes the number of matching second feature points included in the second region. The step of determining the first weight based on the first feature point matching relationship and determining the second weight based on the second feature point matching relationship includes:

[0014] Determine the pixel values ​​of the first region to obtain the first pixel value, and determine the pixel values ​​of the second region to obtain the second pixel value;

[0015] Calculate the number of matches for the first feature point to obtain the first target value, and calculate the number of matches for the second feature point to obtain the second target value;

[0016] The first weight is calculated based on the first pixel value and the first target value, and the second weight is calculated based on the second pixel value and the second target value.

[0017] Optionally, calculating the first weight based on the first pixel value and the first target value, and calculating the second weight based on the second pixel value and the second target value, includes:

[0018] Calculate the ratio of the first sum and the second sum to obtain the third target value, wherein the first sum is the sum of the first target value and the second target value, and the second sum is the sum of the first pixel value and the second pixel value;

[0019] Calculate the product of the third target value and the first pixel value to obtain the fourth target value, and calculate the product of the third target value and the second pixel value to obtain the fifth target value;

[0020] The ratio of the fourth target value to the first target value is calculated to obtain the first weight, and the ratio of the fifth target value to the second target value is calculated to obtain the second weight.

[0021] Optionally, before obtaining the localization result by weighting the sampling probabilities of the first pixel set and the second pixel set according to the first weight and the second weight, the method further includes:

[0022] The sampling probability of the first pixel set is obtained by calculating the ratio of the fourth target value to the first product value, and the sampling probability of the second pixel set is obtained by calculating the ratio of the fifth target value to the second product value.

[0023] Wherein, the first product is the product of the first target value and the third sum, the second product is the product of the second target value and the third sum, and the third sum is the sum of the first target value and the second target value.

[0024] Optionally, the step of weighting the sampling probabilities of the first pixel set and the second pixel set according to the first weight and the second weight to obtain the positioning result includes:

[0025] Determine the normal vectors of multiple triangular meshes in the first region;

[0026] Determine the angles between the coordinate axes of the panoramic image and the normal vectors of the multiple triangular meshes to obtain multiple angles, and the multiple angles correspond one-to-one with the normal vectors of the multiple triangular meshes;

[0027] The multiple included angles are filtered according to preset included angle values ​​to determine at least one target included angle value;

[0028] Using the at least one target included angle value as a residual, the sampling probability of the first pixel set is optimized to obtain the optimized sampling probability of the first pixel set;

[0029] The sampling probabilities of the optimized first pixel set and the second pixel set are weighted and calculated according to the first weight and the second weight to obtain the positioning result.

[0030] Secondly, embodiments of this application also provide an image positioning device, comprising:

[0031] The segmentation module is used to segment the panoramic image into regions to obtain a first region and a second region. The first region is the ground region in the panoramic image, and the second region is the background region in the panoramic image.

[0032] The extraction module is used to extract features from the first region to obtain the first feature point matching relationship, and to extract features from the second region to obtain the second feature point matching relationship;

[0033] The determining module is used to determine a first weight based on the first feature point matching relationship and to determine a second weight based on the second feature point matching relationship, wherein the first weight is the weight value of the sampling probability of the first region and the second weight is the weight value of the sampling probability of the second region;

[0034] The calculation module is used to perform a weighted calculation on the sampling probability of the first region and the sampling probability of the second region according to the first weight and the second weight, so as to obtain the positioning result.

[0035] Thirdly, embodiments of this application also provide an electronic device, including: a transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; the processor is configured to read the program in the memory to implement the steps in the method described in the first aspect above.

[0036] Fourthly, embodiments of this application also provide a readable storage medium for storing a program, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0037] Fifthly, embodiments of this application also provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps of the method as described in the first aspect.

[0038] This application provides a method, apparatus, electronic device, storage medium, and program product for image localization. The method includes: dividing a panoramic image into regions to obtain a first region and a second region, wherein the first region is a ground region in the panoramic image and the second region is a background region in the panoramic image; extracting features from the first region to obtain a first feature point matching relationship, and extracting features from the second region to obtain a second feature point matching relationship; determining a first weight based on the first feature point matching relationship, and determining a second weight based on the second feature point matching relationship, wherein the first weight is a weight value of the sampling probability of the first region, and the second weight is a weight value of the sampling probability of the second region; and performing a weighted calculation on the sampling probabilities of the first region and the second region according to the first weight and the second weight to obtain a localization result. This application improves the localization effect of panoramic images by dividing the panoramic image into different ground and background regions, extracting features from the ground and background regions, determining the weight values ​​of the sampling probabilities of different regions, and performing a weighted calculation to obtain a localization result. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating the image localization method provided in an embodiment of this application;

[0041] Figure 2 This is a schematic diagram of the region division provided in the embodiments of this application;

[0042] Figure 3 This is a schematic diagram of the image positioning device provided in the embodiments of this application;

[0043] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] The terms "first," "second," etc., used in the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. Additionally, the use of "and / or" in this application indicates at least one of the connected objects, such as A and / or B and / or C, representing seven possibilities: including A alone, B alone, C alone, and the presence of both A and B, both B and C, both A and C, and the presence of A, B, and C.

[0046] See Figure 1 , Figure 1 This is a flowchart illustrating the image localization method provided in an embodiment of this application. Figure 1 As shown, the image localization method may include the following steps:

[0047] Step 101: Divide the panoramic image into regions to obtain a first region and a second region. The first region is the ground region in the panoramic image, and the second region is the background region in the panoramic image.

[0048] In this embodiment, the technical solution of this application is applied to Simultaneous Localization and Mapping (SLAM) using a panoramic camera. Specifically, SLAM refers to the process of simultaneously performing self-localization and environment mapping using a panoramic camera. The panoramic image is an image captured by a panoramic camera, which typically refers to a camera capable of capturing a 360-degree scene, thus obtaining a comprehensive view of the surrounding environment.

[0049] It should be noted that in this embodiment, the panoramic image can be segmented using the SAM segmentation model. The SAM segmentation model is an image segmentation model that introduces the prompt paradigm from natural language processing into computer vision. It achieves accurate image segmentation through three interactive methods: click, bounding box selection, and automatic recognition, significantly improving the efficiency of image segmentation. Figure 2 As shown, after dividing the panoramic image into regions using the SAM segmentation model, this embodiment uses three regions as an example for illustration. The first region is the ground region in the panoramic image, the second region is the background region in the panoramic image, and the third region is the sky region.

[0050] It should be noted that in this embodiment, since the depth of field in the sky area is deep and the feature points on the clouds are dynamic points, which will have an adverse effect on the SLAM system, the feature points in the third region are directly removed, so only the first and second regions need to be processed.

[0051] Specifically, the first area can be an area composed of ground, roads, or floors, etc., and the second area can be an area composed of buildings between the sky and the ground.

[0052] Step 102: Extract features from the first region to obtain the first feature point matching relationship, and extract features from the second region to obtain the second feature point matching relationship.

[0053] In this embodiment, feature extraction is performed on different regions. It should be noted that feature extraction may include steps such as feature extraction, feature matching, and feature tracking, which are conventional SLAM techniques and will not be elaborated upon in this embodiment. After feature extraction is performed on the first region and the second region respectively, a first feature point matching relationship and a second feature point matching relationship are generated. The first feature point matching relationship and the second feature point matching relationship are used to determine and track feature points in the first region and the second region.

[0054] Step 103: Determine a first weight based on the first feature point matching relationship, and determine a second weight based on the second feature point matching relationship. The first weight is the weight value of the sampling probability of the first region, and the second weight is the weight value of the sampling probability of the second region.

[0055] In this embodiment, the feature point matching relationship of two regions is obtained. Since they use different algorithms, the uniformity and density of the obtained feature point matching are different. Therefore, in order to balance the influence of the two regions on the pose calculation, a weighted method is used to weight the two feature point matching relationships.

[0056] Specifically, the first weight is determined by the first feature point matching relationship and the second weight is determined by the second feature point matching relationship. It should be noted that the first weight corresponds to the weight value of the sampling probability of the first pixel set, and the second weight corresponds to the weight value of the sampling probability of the second pixel set.

[0057] Step 104: Calculate the sampling probability of the first region and the sampling probability of the second region by weighting according to the first weight and the second weight, and obtain the positioning result.

[0058] In this embodiment, after weighting the first pixel set and the second pixel set using a first weight and a second weight, the position and pose are calculated. The 3D structure of the environment is reconstructed using the matching relationship between feature points across different frames. Finally, an optimization algorithm is used to adjust the camera's motion estimation and the position of map points to reduce reprojection errors and improve overall accuracy, thereby generating a localization result. When using a panoramic camera for visual SLAM, the camera can provide a wider field of view than a traditional single-lens camera, helping to capture environmental information more comprehensively. This wider field of view reduces visual occlusion and blind spots, making the SLAM algorithm more stable during feature matching and map construction.

[0059] This application provides an image localization method, which includes: dividing a panoramic image into regions to obtain a first region and a second region, wherein the first region is a ground region in the panoramic image and the second region is a background region in the panoramic image; extracting features from the first region to obtain a first feature point matching relationship, and extracting features from the second region to obtain a second feature point matching relationship; determining a first weight based on the first feature point matching relationship, and determining a second weight based on the second feature point matching relationship, wherein the first weight is a weight value of the sampling probability of the first region, and the second weight is a weight value of the sampling probability of the second region; and performing a weighted calculation on the sampling probabilities of the first region and the second region according to the first weight and the second weight to obtain a localization result. This application improves the localization effect of panoramic images by dividing the panoramic image into different ground and background regions, extracting features from the ground and background regions, determining the weight values ​​of the sampling probabilities of different regions, and performing a weighted calculation to obtain a localization result.

[0060] In some feasible implementations, optionally, the step of extracting features from the first region to obtain the first feature point matching relationship, and extracting features from the second region to obtain the second feature point matching relationship, includes:

[0061] The first feature tracking algorithm is used to perform image matching on the first feature points included in the first region to obtain the matching relationship of the first feature points;

[0062] The second feature tracking algorithm is used to perform feature matching and feature tracking on the second feature points included in the second region to obtain the matching relationship of the second feature points.

[0063] In this embodiment, for the first region, since most of the objects are static and stable and easy to track, the first feature tracking algorithm, namely the ORB feature algorithm, is still used for matching and tracking in this region. The ORB feature algorithm is a popular feature detection and descriptor extraction algorithm, often used in computer vision tasks, especially in real-time applications, because it is both fast and effective.

[0064] For the second region, due to the weak texture and high similarity of the ground area, ORB feature point matching is not effective. Therefore, a second feature tracking algorithm, namely LoFTR, a matching algorithm for weak textures, is used for image matching between neighboring frames. LoFTR establishes image feature detection, description, and matching at a coarse-grained level, and then refines the sub-pixel level dense matching at a fine-grained level, instead of performing image feature detection, description, and matching sequentially.

[0065] Optionally, the first feature point matching relationship includes the number of matching first feature points included in the first region, and the second feature point matching relationship includes the number of matching second feature points included in the second region. The step of determining the first weight based on the first feature point matching relationship and determining the second weight based on the second feature point matching relationship includes:

[0066] Determine the pixel values ​​of the first region to obtain the first pixel value, and determine the pixel values ​​of the second region;

[0067] Calculate the number of matches for the first feature point to obtain the first target value, and calculate the number of matches for the second feature point to obtain the second target value;

[0068] The first weight is calculated based on the first pixel value and the first target value, and the second weight is calculated based on the second pixel value and the second target value.

[0069] In this embodiment, after obtaining the feature point matching relationship between the two regions, the uniformity and density of the obtained feature point matching relationship are different because the algorithms used in the two regions are different. Therefore, in order to balance the influence of the two regions on pose calculation, the influence is reduced by weighting.

[0070] Specifically, the pixel values ​​of the first region and the second region are first counted to obtain the first pixel value S1 and the second pixel value S2.

[0071] Then, the number of feature point matches in different regions is counted. The number of feature point matches in the first region is N1, and the number of feature point matches in the second region is N2. Therefore, the first weight is calculated using the first pixel value and the first target value, and the second weight is calculated using the second pixel value and the second target value.

[0072] Optionally, calculating the first weight based on the first pixel value and the first target value, and calculating the second weight based on the second pixel value and the second target value, includes:

[0073] Calculate the ratio of the first sum and the second sum to obtain the third target value, wherein the first sum is the sum of the first target value and the second target value, and the second sum is the sum of the first pixel value and the second pixel value;

[0074] Calculate the product of the third target value and the first pixel value to obtain the fourth target value, and calculate the product of the third target value and the second pixel value to obtain the fifth target value;

[0075] The ratio of the fourth target value to the first target value is calculated to obtain the first weight, and the ratio of the fifth target value to the second target value is calculated to obtain the second weight.

[0076] In this embodiment, the third objective value is the number of feature point matches N that should ideally be allocated per unit area. avg Among them, the third objective value N avg It can be calculated using the following formula:

[0077]

[0078] Therefore, assuming a relatively balanced distribution of feature point matching, the number of feature point matches that the first and second regions should possess are the fourth target value N1' and the fifth target value N2', respectively. The fourth and fifth target values ​​can be calculated using the following formulas:

[0079]

[0080] It should be noted that, since actual situations often differ, it is necessary to weight the matching relationship between the two regions so that the sampling probability of the matching relationship between the two regions is different during pose calculation. The weighting value is set as the ratio of the actual matching value to the theoretical matching value, i.e. As the weighted value of the first region matching relationship, i.e., the first weight, As a weighted value for the matching relationship in the second region.

[0081] Optionally, before obtaining the localization result by weighting the sampling probabilities of the first pixel set and the second pixel set according to the first weight and the second weight, the method further includes:

[0082] The sampling probability of the first pixel set is obtained by calculating the ratio of the fourth target value to the first product value, and the sampling probability of the second pixel set is obtained by calculating the ratio of the fifth target value to the second product value.

[0083] Wherein, the first product is the product of the first target value and the third sum, the second product is the product of the second target value and the third sum, and the third sum is the sum of the first target value and the second target value.

[0084] In this embodiment, the third sum N = N1 + N2. When calculating the pose of the matched pairs, K pairs of matched pairs are selected with equal probability (1 / N) for model calculation. Then, the matched pairs that conform to the model are counted. By iterating continuously (usually limiting the number of iterations), the matched pairs that conform to the model are obtained. The model with the most matched pairs conforming to the model (i.e., the most inliers) is decomposed to obtain the pose. Since the matched pairs are selected with equal probability, the actual probability of selecting areas where matched pairs are clustered is relatively high, resulting in the matched pairs used for pose calculation being concentrated locally. The more uniform the distribution of matched pairs used for pose calculation, the more accurate and robust the pose calculation. Therefore, given a fixed distribution of matched pairs, we change the probability of selecting matched pairs to distribute the matched pairs used for pose calculation from another perspective. That is, we weight the sampling probability of matched pairs in different regions according to the weights calculated above. The sampling probability of matched pairs in the first region is... The sampling probability of the second region matching pair is

[0085]

[0086] Optionally, the step of weighting the sampling probabilities of the first pixel set and the second pixel set according to the first weight and the second weight to obtain the positioning result includes:

[0087] Determine the normal vectors of multiple triangular meshes in the first region;

[0088] Determine the angles between the coordinate axes of the panoramic image and the normal vectors of the multiple triangular meshes to obtain multiple angles, and the multiple angles correspond one-to-one with the normal vectors of the multiple triangular meshes;

[0089] The multiple included angles are filtered according to preset included angle values ​​to determine at least one target included angle value;

[0090] Using the at least one target included angle value as a residual, the sampling probability of the first pixel set is optimized to obtain the optimized sampling probability of the first pixel set;

[0091] The sampling probabilities of the optimized first pixel set and the second pixel set are weighted and calculated according to the first weight and the second weight to obtain the positioning result.

[0092] In this embodiment, during the tracking process, ground points typically possess a constraint—planarity—allowing for the acquisition of numerous location surfaces. This constraint can be incorporated into the SLAM system for backend optimization. Specifically, multiple location surfaces are Delaunay triangulated, and the normal vectors of each triangular mesh are determined. Since the camera pose is usually relatively stable during acquisition, constrainable triangular meshes can be filtered based on the angle between the camera's Y-axis orientation and the triangular mesh normal vector. Meshes with an angle greater than a preset angle are discarded; in this embodiment, a preset angle of 15° is used as an example. Finally, the angle between the remaining triangular mesh normal vectors and the Y-axis is used as a residual. During the backend optimization process, this residual is used as a constraint to optimize, resulting in more accurate pose and point cloud data.

[0093] This application improves the positioning effect of panoramic images by dividing panoramic images into different ground and background regions, extracting features from the ground and background regions, determining the weight values ​​of the sampling probabilities of different regions, and performing weighted calculations to obtain the positioning results.

[0094] See Figure 3 , Figure 3 This is a structural diagram of the image positioning device provided in an embodiment of this application. For example... Figure 3 As shown, the image positioning device 300 includes:

[0095] The segmentation module 310 is used to segment the panoramic image into regions to obtain a first region and a second region, wherein the first region is the ground region in the panoramic image and the second region is the background region in the panoramic image.

[0096] The extraction module 320 is used to extract features from the first region to obtain the first feature point matching relationship, and to extract features from the second region to obtain the second feature point matching relationship;

[0097] The determining module 330 is used to determine a first weight based on the first feature point matching relationship and to determine a second weight based on the second feature point matching relationship, wherein the first weight is the weight value of the sampling probability of the first region and the second weight is the weight value of the sampling probability of the second region.

[0098] The calculation module 340 is used to perform a weighted calculation on the sampling probability of the first region and the sampling probability of the second region according to the first weight and the second weight to obtain the positioning result.

[0099] Optionally, the extraction module 320 includes:

[0100] The first matching submodule is used to perform image matching on the first feature points included in the first region using a first feature tracking algorithm to obtain the first feature point matching relationship;

[0101] The second matching submodule is used to perform feature matching and feature tracking on the second feature points included in the second region using the second feature tracking algorithm, so as to obtain the matching relationship of the second feature points.

[0102] Optionally, the first feature point matching relationship includes the number of matching first feature points included in the first region, and the second feature point matching relationship includes the number of matching second feature points included in the second region. The determining module 330 includes:

[0103] The second determining submodule is used to determine the pixel value of the first region to obtain the first pixel value, and to determine the pixel value of the second region;

[0104] The first calculation submodule is used to calculate the number of matches of the first feature point to obtain a first target value, and to calculate the number of matches of the second feature point to obtain a second target value;

[0105] The second calculation submodule is used to calculate the first weight based on the first pixel value and the first target value, and to calculate the second weight based on the second pixel value and the second target value.

[0106] Optionally, the second computational submodule includes:

[0107] The first calculation unit is used to calculate the ratio of the first sum and the second sum to obtain a third target value, wherein the first sum is the sum of the first target value and the second target value, and the second sum is the sum of the first pixel value and the second pixel value;

[0108] The second calculation unit is used to calculate the product of the third target value and the first pixel value to obtain the fourth target value, and to calculate the product of the third target value and the second pixel value to obtain the fifth target value;

[0109] The third calculation unit is used to calculate the ratio of the fourth target value to the first target value to obtain the first weight, and to calculate the ratio of the fifth target value to the second target value to obtain the second weight.

[0110] Optional, also includes:

[0111] The probability calculation module is used to calculate the ratio of the fourth target value and the first product value to obtain the sampling probability of the first pixel set, and to calculate the ratio of the fifth target value and the second product value to obtain the sampling probability of the second pixel set.

[0112] Wherein, the first product is the product of the first target value and the third sum, the second product is the product of the second target value and the third sum, and the third sum is the sum of the first target value and the second target value.

[0113] Optionally, the computing module 340 includes:

[0114] The third determining submodule is used to determine the normal vectors of multiple triangular meshes in the first region;

[0115] The fourth determining submodule is used to determine the angle between the coordinate axes of the panoramic image and the normal vectors of the multiple triangular meshes, thereby obtaining multiple angles, and the multiple angles correspond one-to-one with the normal vectors of the multiple triangular meshes;

[0116] The filtering submodule is used to filter the multiple angles according to preset angle values ​​to determine at least one target angle value;

[0117] An optimization submodule is used to optimize the sampling probability of the first pixel set by using the at least one target included angle value as a residual, so as to obtain the optimized sampling probability of the first pixel set.

[0118] The weighted calculation submodule is used to perform weighted calculation on the sampling probability of the optimized first pixel set and the sampling probability of the second pixel set according to the first weight and the second weight, so as to obtain the positioning result.

[0119] This application improves the positioning effect of panoramic images by dividing panoramic images into different ground and background regions, extracting features from the ground and background regions, determining the weight values ​​of the sampling probabilities of different regions, and performing weighted calculations to obtain the positioning results.

[0120] This application also provides a communication device. Please refer to [link to relevant documentation]. Figure 4 The communication device may include a processor 401, a memory 402, and a program 4021 stored in the memory 402 and capable of running on the processor 401.

[0121] When the communication device is an electronic device, program 4021 can be executed by processor 401. Figure 1 Any step in the corresponding method embodiment:

[0122] The panoramic image is divided into regions to obtain a first region and a second region. The first region is the ground region in the panoramic image, and the second region is the background region in the panoramic image.

[0123] Feature extraction is performed on the first region to obtain the first feature point matching relationship, and feature extraction is performed on the second region to obtain the second feature point matching relationship;

[0124] A first weight is determined based on the first feature point matching relationship, and a second weight is determined based on the second feature point matching relationship. The first weight is the weight value of the sampling probability of the first region, and the second weight is the weight value of the sampling probability of the second region.

[0125] The sampling probabilities of the first region and the second region are weighted and calculated based on the first weight and the second weight to obtain the positioning result.

[0126] Optionally, the step of extracting features from the first region to obtain the first feature point matching relationship, and extracting features from the second region to obtain the second feature point matching relationship, includes:

[0127] The first feature tracking algorithm is used to perform image matching on the first feature points included in the first region to obtain the matching relationship of the first feature points;

[0128] The second feature tracking algorithm is used to perform feature matching and feature tracking on the second feature points included in the second region to obtain the matching relationship of the second feature points.

[0129] Optionally, the first feature point matching relationship includes the number of matching first feature points included in the first region, and the second feature point matching relationship includes the number of matching second feature points included in the second region. The step of determining the first weight based on the first feature point matching relationship and determining the second weight based on the second feature point matching relationship includes:

[0130] Determine the pixel values ​​of the first region to obtain the first pixel value, and determine the pixel values ​​of the second region to obtain the second pixel value;

[0131] Calculate the number of matches for the first feature point to obtain the first target value, and calculate the number of matches for the second feature point to obtain the second target value;

[0132] The first weight is calculated based on the first pixel value and the first target value, and the second weight is calculated based on the second pixel value and the second target value.

[0133] Optionally, calculating the first weight based on the first pixel value and the first target value, and calculating the second weight based on the second pixel value and the second target value, includes:

[0134] Calculate the ratio of the first sum and the second sum to obtain the third target value, wherein the first sum is the sum of the first target value and the second target value, and the second sum is the sum of the first pixel value and the second pixel value;

[0135] Calculate the product of the third target value and the first pixel value to obtain the fourth target value, and calculate the product of the third target value and the second pixel value to obtain the fifth target value;

[0136] The ratio of the fourth target value to the first target value is calculated to obtain the first weight, and the ratio of the fifth target value to the second target value is calculated to obtain the second weight.

[0137] Optionally, before obtaining the localization result by weighting the sampling probabilities of the first pixel set and the second pixel set according to the first weight and the second weight, the method further includes:

[0138] The sampling probability of the first pixel set is obtained by calculating the ratio of the fourth target value to the first product value, and the sampling probability of the second pixel set is obtained by calculating the ratio of the fifth target value to the second product value.

[0139] Wherein, the first product is the product of the first target value and the third sum, the second product is the product of the second target value and the third sum, and the third sum is the sum of the first target value and the second target value.

[0140] Optionally, the step of weighting the sampling probabilities of the first pixel set and the second pixel set according to the first weight and the second weight to obtain the positioning result includes:

[0141] Determine the normal vectors of multiple triangular meshes in the first region;

[0142] Determine the angles between the coordinate axes of the panoramic image and the normal vectors of the multiple triangular meshes to obtain multiple angles, and the multiple angles correspond one-to-one with the normal vectors of the multiple triangular meshes;

[0143] The multiple included angles are filtered according to preset included angle values ​​to determine at least one target included angle value;

[0144] Using the at least one target included angle value as a residual, the sampling probability of the first pixel set is optimized to obtain the optimized sampling probability of the first pixel set;

[0145] The sampling probabilities of the optimized first pixel set and the second pixel set are weighted and calculated according to the first weight and the second weight to obtain the positioning result.

[0146] This application improves the positioning effect of panoramic images by dividing panoramic images into different ground and background regions, extracting features from the ground and background regions, determining the weight values ​​of the sampling probabilities of different regions, and performing weighted calculations to obtain the positioning results.

[0147] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described panoramic image positioning method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0148] This application also provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described panoramic image positioning method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0149] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0150] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0151] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for locating an image, characterized in that, The method includes: The panoramic image is divided into regions to obtain a first region and a second region. The first region is the ground region in the panoramic image, and the second region is the background region in the panoramic image. Feature extraction is performed on the first region to obtain a first feature point matching relationship, and feature extraction is performed on the second region to obtain a second feature point matching relationship; A first weight is determined based on the first feature point matching relationship, and a second weight is determined based on the second feature point matching relationship. The first weight is the weight value of the sampling probability of the first region, and the second weight is the weight value of the sampling probability of the second region. The sampling probabilities of the first region and the second region are weighted and calculated based on the first weight and the second weight to obtain the positioning result.

2. The method according to claim 1, characterized in that, The step of extracting features from the first region to obtain the first feature point matching relationship, and extracting features from the second region to obtain the second feature point matching relationship, includes: The first feature tracking algorithm is used to perform image matching on the first feature points included in the first region to obtain the matching relationship of the first feature points; The second feature tracking algorithm is used to perform feature matching and feature tracking on the second feature points included in the second region to obtain the matching relationship of the second feature points.

3. The method according to claim 1, characterized in that, The first feature point matching relationship includes the number of matching first feature points included in the first region, and the second feature point matching relationship includes the number of matching second feature points included in the second region. The step of determining a first weight based on the first feature point matching relationship and determining a second weight based on the second feature point matching relationship includes: Determine the pixel values ​​of the first region to obtain the first pixel value, and determine the pixel values ​​of the second region to obtain the second pixel value; Calculate the number of matches for the first feature point to obtain the first target value, and calculate the number of matches for the second feature point to obtain the second target value; The first weight is calculated based on the first pixel value and the first target value, and the second weight is calculated based on the second pixel value and the second target value.

4. The method according to claim 3, characterized in that, The step of calculating the first weight based on the first pixel value and the first target value, and calculating the second weight based on the second pixel value and the second target value, includes: Calculate the ratio of the first sum and the second sum to obtain the third target value, wherein the first sum is the sum of the first target value and the second target value, and the second sum is the sum of the first pixel value and the second pixel value; Calculate the product of the third target value and the first pixel value to obtain the fourth target value, and calculate the product of the third target value and the second pixel value to obtain the fifth target value; The ratio of the fourth target value to the first target value is calculated to obtain the first weight, and the ratio of the fifth target value to the second target value is calculated to obtain the second weight.

5. The method according to claim 4, characterized in that, Before obtaining the positioning result by weighting the sampling probabilities of the first region and the second region according to the first weight and the second weight, the method further includes: The sampling probability of the first region is obtained by calculating the ratio of the fourth target value to the first product value, and the sampling probability of the second region is obtained by calculating the ratio of the fifth target value to the second product value. Wherein, the first product is the product of the first target value and the third sum, the second product is the product of the second target value and the third sum, and the third sum is the sum of the first target value and the second target value.

6. The method according to claim 5, characterized in that, The step of weighting the sampling probabilities of the first region and the second region according to the first weight and the second weight to obtain the positioning result includes: Determine the normal vectors of multiple triangular meshes in the first region; Determine the angles between the coordinate axes of the panoramic image and the normal vectors of the multiple triangular meshes to obtain multiple angles, and the multiple angles correspond one-to-one with the normal vectors of the multiple triangular meshes; The multiple included angles are filtered according to preset included angle values ​​to determine at least one target included angle value; Using the at least one target included angle value as a residual, the sampling probability of the first region is optimized to obtain the optimized sampling probability of the first region; The sampling probabilities of the optimized first region and the second region are weighted and calculated according to the first weight and the second weight to obtain the positioning result.

7. An image positioning device, characterized in that, The device includes: The segmentation module is used to segment the panoramic image into regions to obtain a first region and a second region. The first region is the ground region in the panoramic image, and the second region is the background region in the panoramic image. The extraction module is used to extract features from the first region to obtain a first feature point matching relationship, and to extract features from the second region to obtain a second feature point matching relationship; The determining module is used to determine a first weight based on the first feature point matching relationship and to determine a second weight based on the second feature point matching relationship, wherein the first weight is the weight value of the sampling probability of the first region and the second weight is the weight value of the sampling probability of the second region; The calculation module is used to perform a weighted calculation on the sampling probability of the first region and the sampling probability of the second region according to the first weight and the second weight, so as to obtain the positioning result.

8. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is configured to read the program in the memory to implement the steps in the image positioning method as described in any one of claims 1 to 6.

9. A readable storage medium for storing a program, characterized in that, When the program is executed by the processor, it implements the steps in the image localization method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product is stored in a storage medium and is executed by at least one processor to implement the steps in the image localization method as described in any one of claims 1 to 6.

Images