Spherical panoramic image feature matching method and system, terminal and storage medium

Abstract

The invention discloses a spherical panoramic image feature matching method and system, a terminal and a storage medium, and the method comprises the steps: obtaining a spherical panoramic image, and carrying out the preprocessing of the spherical panoramic image; screening the preprocessed spherical panoramic image based on a spatial constraint model, and constructing a scene graph based on the screened spherical panoramic image; segmenting the scene graph into a plurality of sub-scene graphs, and performing priority ranking on the sub-scene graphs based on a graph sequence scheduling algorithm; and performing geometric correction and feature extraction on all the screened spherical panoramic images, and executing a feature matching task of the spherical panoramic images in the sub-scene images according to the priority ranking of the sub-scene images. Through a spatial constraint model, a graph order scheduling algorithm, geometric correction and a cascade Hash matching algorithm, the problem that feature matching efficiency and precision are difficult to consider when a spherical image feature matching technology processes large-scale high-resolution spherical images is solved.

Description

Technical Field

[0001] The present invention relates to the field of photogrammetry and computer vision technology, and in particular to a spherical panoramic image feature matching method, system, terminal and storage medium. Background Art

[0002] 3D scene reconstruction and real-time positioning technologies are crucial in fields such as autonomous driving and smart cities. Perspective imaging is the primary data source for traditional scene 3D reconstruction and real-time positioning technologies, and feature matching is a prerequisite for image-based 3D reconstruction. However, current perspective cameras used to capture perspective images face challenges with continuous tracking and high data acquisition costs, leading to increased interest in spherical imaging technologies.

[0003] Spherical imaging technology, with its 360° full-view coverage, can serve as an important data source for 3D scene reconstruction and real-time positioning. However, existing spherical image feature matching technologies face numerous challenges: Geometric distortion affects the accuracy of feature point matching when traditional feature extraction algorithms process large-scale, high-resolution spherical images. Brute-force matching is difficult to scale to large datasets. Hash indexing methods struggle to balance efficiency and accuracy due to hash collisions. Furthermore, existing methods fail to fully exploit the parallel acceleration potential of GPUs, resulting in low memory utilization and limited hardware performance.

[0004] Therefore, the existing spherical image feature matching technology has the problem of difficulty in balancing the efficiency and accuracy of feature matching when processing large-scale high-resolution spherical images. Summary of the Invention

[0005] The main purpose of the present invention is to provide a spherical panoramic image feature matching method, system, terminal and storage medium, aiming to solve the problem in the existing spherical image feature matching technology that it is difficult to strike a balance between efficiency and accuracy in feature matching when processing large-scale high-resolution spherical images.

[0006] To achieve the above object, the present invention provides a spherical panoramic image feature matching method, which includes the following steps:

[0007] Acquire a spherical panoramic image, and pre-process the spherical panoramic image;

[0008] screening the pre-processed spherical panoramic image based on a spatial constraint model, and constructing a scene graph based on the screened spherical panoramic image;

[0009] Splitting the scene graph into multiple sub-scene graphs, and prioritizing the sub-scene graphs based on a graph order scheduling algorithm;

[0010] Performing geometric correction and feature extraction on all the screened spherical panoramic images, and performing feature matching tasks on the spherical panoramic images in the sub-scene graphs according to the priority order of the sub-scene graphs;

[0011] Output spherical panoramic image feature matching results.

[0012] Optionally, screening the pre-processed spherical panoramic image based on the spatial constraint model and constructing a scene graph based on the screened spherical panoramic image includes:

[0013] Determining a dynamic distance threshold of the spatial constraint model, and screening the pre-processed spherical panoramic image based on the dynamic distance threshold;

[0014] Determining a posture angle difference threshold of the spatial constraint model, and performing a secondary screening on the spherical panoramic image after the primary screening based on the posture angle difference threshold;

[0015] Based on the secondary filtered spherical panoramic image, a scene graph is constructed.

[0016] Optionally, determining a dynamic distance threshold of the spatial constraint model and screening the pre-processed spherical panoramic image based on the dynamic distance threshold includes:

[0017] Determining a dynamic distance threshold of the spatial constraint model, wherein the dynamic distance threshold includes at least a first distance threshold and a second distance threshold;

[0018] Calculating the Euclidean distance between the current pre-processed spherical panoramic image and other pre-processed spherical panoramic images;

[0019] Based on the scene density of the location of the current pre-processed spherical panoramic image, a corresponding dynamic distance threshold is selected for screening to obtain a screened spherical panoramic image.

[0020] Optionally, determining a posture angle difference threshold of the spatial constraint model, and performing a secondary screening on the spherical panoramic image after the primary screening based on the posture angle difference threshold, comprises:

[0021] Determining an attitude angle difference threshold of the spatial constraint model;

[0022] Calculating the attitude angle difference between the current filtered spherical panoramic image and the other filtered spherical panoramic images to obtain an attitude angle difference value;

[0023] The attitude angle difference is compared with the attitude angle difference threshold, and the spherical panoramic image corresponding to the attitude angle difference that is too large is screened out to obtain a spherical panoramic image after secondary screening.

[0024] Optionally, dividing the scene graph into a plurality of sub-scene graphs and prioritizing the sub-scene graphs based on a graph order scheduling algorithm includes:

[0025] Splitting the scene graph into a plurality of sub-scene graphs based on a video memory constraint; wherein the sub-scene graphs include at least two of the filtered spherical panoramic images;

[0026] Calculating the degree of overlap between the current subscene graph and other subscene graphs in the scene graph; wherein the degree of overlap is defined as the number of shared images between the current subscene graph and other subscene graphs in the scene graph;

[0027] Prioritize other sub-scene graphs in the scene graph based on the overlap.

[0028] Optionally, performing geometric correction on all the screened spherical panoramic images includes:

[0029] According to all the filtered spherical panoramic images, obtaining pixel coordinates corresponding to the spherical panoramic images on the cube plane;

[0030] Calculating the orientation vector of the pixel coordinates in the pinhole camera coordinate system and converting it into the orientation vector of the spherical camera coordinate system;

[0031] Obtain the spherical coordinates of the pixel based on the orientation vector of the spherical camera coordinate system;

[0032] A pixel value of the spherical coordinate is obtained based on bilinear interpolation, and the pixel value is assigned to the pixel coordinate.

[0033] Optionally, the feature matching task is based on a cascade hash matching algorithm, specifically including:

[0034] Obtaining feature descriptors for extracting features from all the filtered spherical panoramic images, and converting the feature descriptors into short binary hash codes;

[0035] Constructing a hash lookup table based on the short binary hash code and the locality sensitive hash function, performing a multi-table hash search based on the query point in the hash lookup table, and returning a first candidate matching point;

[0036] remapping the short binary hash code into a binary hash code, and returning a second candidate matching point based on a Hamming distance between the query point and the first candidate matching point;

[0037] Determining a nearest neighbor and a second nearest neighbor based on the Euclidean distance between the query point and the second candidate matching point, and performing a ratio test on the nearest neighbor using the second nearest neighbor;

[0038] Based on the nearest neighbors that pass the ratio test, a feature matching relationship between spherical panoramic images is established.

[0039] In addition, to achieve the above-mentioned purpose, the present invention further provides a spherical panoramic image feature matching system, wherein the spherical panoramic image feature matching system includes:

[0040] An image acquisition module, used for acquiring a spherical panoramic image and preprocessing the spherical panoramic image;

[0041] A scene construction module is used to filter the pre-processed spherical panoramic images according to the spatial constraint model and construct a scene graph based on the filtered spherical panoramic images;

[0042] A scene sorting module, configured to divide the scene graph into a plurality of sub-scene graphs and prioritize the sub-scene graphs based on a graph sequence scheduling algorithm;

[0043] A feature matching module is used to perform geometric correction and feature extraction on all the screened spherical panoramic images, and perform feature matching tasks on the spherical panoramic images in the sub-scene graphs according to the priority order of the sub-scene graphs;

[0044] The result output module is used to output the spherical panoramic image feature matching results.

[0045] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a spherical panoramic image feature matching program stored on the memory and runnable on the processor, and when the spherical panoramic image feature matching program is executed by the processor, the steps of the spherical panoramic image feature matching method as described above are implemented.

[0046] In addition, to achieve the above-mentioned purpose, the present invention also provides a storage medium, wherein the storage medium is a computer-readable storage medium, and the computer-readable storage medium stores a spherical panoramic image feature matching program, and when the spherical panoramic image feature matching program is executed by the processor, the steps of the spherical panoramic image feature matching method as described above are implemented.

[0047] In the present invention, a spherical panoramic image is acquired and preprocessed; the preprocessed spherical panoramic image is screened based on a spatial constraint model, and a scene graph is constructed based on the screened spherical panoramic image; the scene graph is divided into multiple sub-scene graphs, and the sub-scene graphs are prioritized based on a graph order scheduling algorithm; geometric correction and feature extraction are performed on all the screened spherical panoramic images, and feature matching tasks for the spherical panoramic images in the sub-scene graphs are performed according to the priority ranking of the sub-scene graphs. This method can solve the problem of spherical image feature matching technology in balancing feature matching efficiency and accuracy when processing large-scale, high-resolution spherical images.

[0048] By using the spatial constraint model to filter the preprocessed spherical panoramic images, a primary screening based on the dynamic distance can retain spatially adjacent spherical panoramic images; a secondary screening based on the attitude angle difference threshold can filter out spherical panoramic images with excessively large attitude angle differences, thereby ensuring the spatial proximity of the images and reducing invalid matching calculations.

[0049] By using a graph-order scheduling algorithm to segment the sub-scene graph and prioritize it, the global matching task can be decomposed into memory-adaptive sub-task streams, taking into account both computational efficiency and resource utilization.

[0050] Through geometric correction, geometric distortion can be reduced and the accuracy of feature extraction can be improved, achieving efficient feature extraction and matching; using cascade hash matching, the feature matching relationship between images can be efficiently established, taking into account both the efficiency and accuracy of feature matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of a preferred embodiment of the spherical panoramic image feature matching method of the present invention;

[0052] Figure 2 It is a mapping relationship diagram from object space three-dimensional points to two-dimensional image points in the spherical panoramic image feature matching method of the present invention;

[0053] Figure 3 3D vector diagram of pixel coordinates converted into camera coordinate system in the spherical panoramic image feature matching method of the present invention;

[0054] Figure 4 Schematic diagram of the effective pixel area in the spherical panoramic image feature matching method of the present invention;

[0055] Figure 5 Schematic diagram of the cascade hash matching process in the spherical panoramic image feature matching method of the present invention;

[0056] Figure 6 1 is a structural diagram of a preferred embodiment of the spherical panoramic image feature matching system of the present invention;

[0057] Figure 7 FIG. 4 is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] With the rapid development of intelligent computing technologies such as the Internet of Things, cloud computing, big data, and spatial geographic information integration, the concept of smart cities has emerged. Scene 3D reconstruction and real-time positioning technologies are extremely important in areas such as autonomous driving and smart cities. 3D scene reconstruction enables the creation and expression of the objective world in virtual reality, while real-time scene positioning provides accurate positioning and navigation services in complex urban environments. Perspective imaging is the primary data source for traditional scene 3D reconstruction and real-time positioning technologies, and feature matching is a prerequisite for image-based 3D reconstruction. However, the perspective cameras used to obtain perspective images face challenges with continuous tracking and high data acquisition costs, leading to increased attention on spherical imaging technology.

[0060] Spherical panoramic images are typically acquired using spherical panoramic cameras, which offer 360° full-view coverage. These images serve as a crucial data source for current scene 3D reconstruction and real-time localization technologies. Spherical panoramic images have different storage methods and camera imaging models, typically represented using equirectangular projection (ERP) for ease of storage and processing. The imaging model of panoramic cameras also describes the mapping relationship from 3D object points to 2D image points. However, existing spherical image feature matching technologies face numerous challenges. Traditional feature extraction algorithms (such as SIFT and SuperPoint) experience uneven distribution of feature points when processing large-scale, high-resolution spherical images due to geometric distortion, significantly reducing matching accuracy. Furthermore, brute-force matching methods have a high time complexity and are difficult to scale to large datasets. While existing hash indexing methods (such as LSH and vocabulary trees) can improve matching efficiency, they are prone to hash collisions in high-dimensional feature spaces, leading to increased false matches and a struggle to balance efficiency and accuracy. Furthermore, existing methods often rely on CPU serial computation and fail to fully exploit the parallel acceleration potential of GPUs, resulting in low memory utilization and difficulty meeting real-time requirements.

[0061] While previous research has attempted to mitigate geometric distortion through spherical projection correction (such as cube mapping and tangent plane projection), two major limitations remain: first, the computational overhead of projection correction is high, making it difficult to meet real-time requirements; second, even after correction, it still relies on highly complex matching algorithms, resulting in overall low efficiency. Furthermore, existing methods still lack optimal scheduling of video memory resources when processing large amounts of data, resulting in insufficient GPU utilization and difficulty in fully utilizing hardware performance.

[0062] Therefore, the existing spherical image feature matching technology has the problem of difficulty in balancing the efficiency and accuracy of feature matching when processing large-scale high-resolution spherical images.

[0063] In response to one or more of the above problems, the present invention obtains a spherical panoramic image and preprocesses the spherical panoramic image; filters the preprocessed spherical panoramic image based on a spatial constraint model, and constructs a scene graph based on the filtered spherical panoramic image; divides the scene graph into multiple sub-scene graphs, and prioritizes the sub-scene graphs based on a graph sequence scheduling algorithm; performs geometric correction and feature extraction on all the filtered spherical panoramic images, and executes feature matching tasks for the spherical panoramic images in the sub-scene graphs according to the priority ranking of the sub-scene graphs.

[0064] The spherical panoramic image feature matching method described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the method includes the following steps:

[0065] Step S10: Acquire a spherical panoramic image and pre-process the spherical panoramic image.

[0066] It should be noted that, in the present invention, the spherical panoramic image feature matching method is for spherical panoramic images obtained by a spherical panoramic camera, wherein the spherical panoramic camera is also called a 360 camera. The spherical panoramic camera can obtain images with an all-round field of view and is widely used in technical fields such as security monitoring, street view maps, and robot navigation. Compared with ordinary planar perspective images, spherical panoramic images have different storage methods and camera imaging models, and can serve as an important data source for three-dimensional reconstruction. Usually, spherical panoramic images are represented by equidistant cylindrical projection (ERP) for easy storage and processing. In the present invention, all spherical panoramic images obtained use the same storage method and are based on the same camera imaging model.

[0067] Furthermore, the acquiring of the spherical panoramic image and preprocessing of the spherical panoramic image specifically include:

[0068] Step S11: Acquire all spherical panoramic images according to the requirements of spherical panoramic image feature matching.

[0069] Specifically, the requirements for spherical panoramic image feature matching need to include at least: the shooting location of the spherical panoramic image to be matched, selecting all spherical panoramic cameras in the location based on the shooting location of the spherical panoramic image to be matched, and collecting corresponding spherical panoramic image data from the spherical panoramic cameras.

[0070] Step S12: Processing the spherical panoramic image using equidistant cylindrical projection.

[0071] like Figure 2 As shown, Figure 2 The figure below is a schematic diagram of the conversion between spherical panoramic images and equidistant cylindrical projection images. Based on the imaging model of a spherical panoramic camera, equidistant cylindrical projection is used to process spherical panoramic images. The imaging model of a panoramic camera describes the mapping relationship from 3D object points to 2D image points. The model is divided into two steps:

[0072] (1) Mapping of object space 3D points to spherical projection points, such as Figure 2 As shown in (a), the object space 3D point P W Projected onto the point p on the sphere;

[0073] (2) Mapping of spherical projection points to two-dimensional image points, such as Figure 2 As shown in (b), the spherical projection point p is further mapped to a point (I x ,I y ).

[0074] Specifically, the spherical projection point can be expressed in terms of the spherical longitude and latitude coordinate system. Or expressed in the spherical rectangular coordinate system O(x,y,z), the latitude and longitude coordinates of the projection point p are The conversion relationship between the rectangular coordinates p(x, y, z) is shown in the following formula (the radius of the sphere is set to 1):

[0075]

[0076] Among them, the Z axis of the spherical rectangular coordinate system O(x, y, z) points to the principal point of the two-dimensional image, which is the intersection of the perpendicular line between the photography center and the image plane and the image plane. The Y axis is vertically downward. The intersection line of the great circle passing through the center point O of the sphere and the projection point p with the equatorial plane is l. The longitude θ of the projection point p is defined as the angle between the intersection line l and the Z axis. The latitude of the projection point p is It is defined as the angle between the intersection line l and the line Op connecting the center O to the projection point p.

[0077] according to Figure 2 (b) shows the spherical longitude and latitude coordinates and the two-dimensional image plane coordinates p(I x ,I y) can be expressed by the following formula:

[0078]

[0079] Where W and H are the width and height of the image respectively, (c x ,c y ) are the coordinates of the principal point of the image.

[0080] Assuming that in the world coordinate system, the posture and position of the panoramic image are represented by the rotation matrix R and the translation vector T respectively, then the object space three-dimensional point P W The corresponding spherical projection point p can be obtained by the following formula:

[0081]

[0082] Step S20: screening the pre-processed spherical panoramic image based on the spatial constraint model, and constructing a scene graph based on the screened spherical panoramic image.

[0083] It should be noted that the pre-processed spherical panoramic image is a spherical panoramic image processed by equidistant cylindrical projection in step S11 , and the pre-processed spherical panoramic image is stored in ERP format.

[0084] Step S21 : determining a dynamic distance threshold of the spatial constraint model, and performing a screening on the pre-processed spherical panoramic image based on the dynamic distance threshold.

[0085] Step S211: Determine a dynamic distance threshold of the spatial constraint model; wherein the dynamic distance threshold includes at least a first distance threshold and a second distance threshold.

[0086] Specifically, the spatial constraint model is proposed to address the redundant matching problem caused by the wide viewing angle characteristics in spherical panoramic image feature matching. The model proposes a spatial proximity measurement method based on the image POS (Position and Orientation System) information, and screens spatially adjacent image pairs through a dynamic distance threshold to reduce invalid matching calculations.

[0087] Optionally, a dynamic distance threshold d th Adaptive settings based on scene density.

[0088] In one embodiment of the present invention, the dynamic distance threshold includes at least a first distance threshold and a second distance threshold. The first distance threshold is defined as the dynamic distance threshold for dense urban scenes, and the second distance threshold is defined as the dynamic distance threshold for sparse suburban scenes.

[0089] For example, in densely populated urban areas, setting a dynamic distance threshold d th= 20m; in sparse suburban areas, set the dynamic distance threshold d th =40m, only retain the spatial distance less than the dynamic distance threshold d th The image pairs enter the subsequent matching process.

[0090] Step S212: Calculate the Euclidean distance between the current pre-processed spherical panoramic image and other pre-processed spherical panoramic images.

[0091] It should be noted that Euclidean distance is the straight-line distance between two points in multidimensional space. It is also one of the most common distance metrics and is widely used in fields such as data analysis, machine learning, and pattern recognition. In this application, the Euclidean distance cannot be directly calculated for a single spherical panoramic image. It is necessary to calculate the Euclidean distance by constructing image pairs. During the screening process, the constructed image pairs can be filtered first, and then the target spherical panoramic image is obtained based on the filtered image pairs.

[0092] Specifically, the GPS coordinates of the current pre-processed spherical panoramic image are obtained (the coordinates are based on the WGS-84 latitude and longitude coordinate system). The GPS coordinates can accurately reflect the specific position of the current pre-processed spherical panoramic image and are used to calculate the Euclidean distance with other pre-processed spherical panoramic images. However, in the actual calculation process, the use of GPS coordinates to calculate the Euclidean distance is based on the distance calculation of the sphere, and the calculation method is relatively complicated. Therefore, in this application, the GPS coordinates of all pre-processed spherical panoramic images are first converted to coordinates in the local Cartesian coordinate system (such as UTM projection) to avoid the use of complex spherical distance calculation methods. The coordinate conversion formula is as follows:

[0093]

[0094] The GPS coordinates of the spherical panoramic image I are I(lon, lat, alt), the local coordinate system coordinates of I are I(x, y, z), the coordinates of the origin of the local coordinate system are O(lon0, lat0), and R e is the radius of the Earth.

[0095] After converting the GPS coordinates of all pre-processed spherical panoramic images into coordinates in the local Cartesian coordinate system, the Euclidean distance between the current pre-processed spherical panoramic image and other pre-processed spherical panoramic images is calculated.

[0096] Furthermore, an image pair is constructed, and the image pair includes the current pre-processed spherical panoramic image and any other pre-processed spherical panoramic image, that is, a pre-processed spherical panoramic image can construct multiple image pairs. For example, the constructed image pair (I i ,I j) contains spherical panoramic image I i and spherical panoramic image I j .

[0097] Therefore, the calculation process of each Euclidean distance can be regarded as the calculation process of the Euclidean distance of the two pre-processed spherical panoramic images contained in each image pair, that is, the Euclidean distance d of the image pair pos (I i ,I j ) is defined as a spherical panoramic image I i and spherical panoramic image I j The Euclidean distance d between pos , this distance can reflect the spatial proximity between images, that is, spatial proximity, and the Euclidean distance d pos The smaller the distance, the higher the possibility of spatial overlap between images. pos The calculation formula is as follows:

[0098]

[0099] Among them, the spherical panoramic image I i The local coordinate system coordinates are I i (x i ,y i ,z i ), spherical panoramic image I j The local coordinate system coordinates are I j (x j ,y j ,z j ).

[0100] Step S213: Based on the scene density of the location of the current pre-processed spherical panoramic image, a corresponding dynamic distance threshold is selected for screening to obtain a screened spherical panoramic image.

[0101] Optionally, set the dynamic distance threshold d th Based on different scenarios, in dense urban areas, d th =20m; in sparse suburban scenes, d th =40m.

[0102] Based on the scene density of the current pre-processed spherical panoramic image, the corresponding dynamic distance threshold is selected for screening. For example, if the scene density of the current pre-processed spherical panoramic image satisfies the scene density of the dense urban scene, the first distance threshold d is selected. th = 20m; if the scene density of the location of the current pre-processed spherical panoramic image meets the scene density of the sparse suburban scene, then the second distance threshold d is selected th=40m; filter based on the selected dynamic distance threshold, and only retain the spatial distance less than the dynamic distance threshold d th The image pairs enter the subsequent matching process, and the screening formula is as follows:

[0103]

[0104] Among them, S ij The value of is used to represent the image pair (I i ,I j ) whether to enter the subsequent matching process, when S ij When the value is 1, it means that the image pair (I i ,I j ) can enter the subsequent matching process, when S ij When the value is 0, it means that the image pair (I i ,I j ) is screened out and does not enter the subsequent matching process.

[0105] The dynamic distance threshold based on the spatial constraint model can filter out all image pairs whose Euclidean distance is less than the dynamic distance threshold. That is, all the spherical panoramic images after one screening can be obtained based on the screened image pairs for entering the subsequent matching process.

[0106] In practical applications, there may be repeated pre-processed spherical panoramic images in all the filtered image pairs obtained, so after one screening, all the filtered spherical panoramic images can also be de-duplicated.

[0107] Step S22: determining a posture angle difference threshold of the spatial constraint model, and performing a secondary screening on the spherical panoramic image after the primary screening based on the posture angle difference threshold.

[0108] Step S221: Determine the attitude angle difference threshold of the spatial constraint model.

[0109] It should be noted that in step S21, the spherical panoramic images are only filtered based on the dynamic distance threshold. Although it can exclude image pairs with a large spatial distance, it cannot exclude image pairs that are spatially close but have large attitude angle differences. In the spatial constraint model, in addition to the dynamic distance threshold, the attitude angle difference of the image can also be constrained. By setting the attitude angle difference threshold, the spherical panoramic images after the first screening can be further filtered. The attitude angle information includes the pitch angle θ′, the azimuth angle φ, and the roll angle ψ. In this application, the attitude angle difference threshold is set based on the pitch angle θ′ and the azimuth angle φ, and does not involve the roll angle ψ.

[0110] Optionally, the attitude angle difference threshold is set to the pitch angle difference threshold Δθ′ th =10°, azimuth difference threshold Δφth =15°.

[0111] Step S222: Calculate the difference in attitude angle between the current filtered spherical panoramic image and the other filtered spherical panoramic images.

[0112] It should be noted that a single spherical panoramic image cannot directly calculate the attitude angle difference. It is also necessary to calculate the attitude angle difference by constructing image pairs. When screening, the constructed image pairs can be filtered first, and then the target spherical panoramic image can be obtained based on the filtered image pairs.

[0113] Furthermore, an image pair is constructed, and the image pair includes the current spherical panoramic image after one screening and any other spherical panoramic image after one screening. That is, a spherical panoramic image after one screening can construct multiple image pairs. For example, the constructed image pair (I i ,I j ) contains spherical panoramic image I i and spherical panoramic image I j .

[0114] In one embodiment of the present invention, step S21 constructs an image pair, which includes two pre-processed spherical panoramic images, and all spherical panoramic images are deduplicated after one screening. All deduplicated image pairs can be obtained based on all deduplicated spherical panoramic images, and the attitude angle differences of these image pairs can be directly calculated in subsequent steps, and there is no need to deduplicate all spherical panoramic images after the second screening.

[0115] Therefore, the calculation process of each attitude angle difference can be regarded as the attitude angle difference calculation process of the two spherical panoramic images after the first screening contained in each image pair. In this application, the pitch angle θ′ and azimuth angle φ in the attitude angle information are used, that is, the pitch angle difference Δθ′ (I i ,I j ) is defined as a spherical panoramic image I i and spherical panoramic image I j The pitch angle difference Δθ′ between the two images is also the azimuth angle difference Δφ′ (I i ,I j ) is defined as image I i and Image I j The azimuth angle difference Δφ′ between the two images. The pitch angle difference Δθ′ and the azimuth angle difference Δφ′ can reflect the difference in viewing angle between the images. The calculation formula is as follows:

[0116]

[0117] Step S223 : comparing the attitude angle difference with the attitude angle difference threshold, filtering out spherical panoramic images corresponding to attitude angle differences that are too large, and obtaining a spherical panoramic image after secondary screening.

[0118] Among them, the spherical panoramic image corresponding to the excessively large attitude angle difference can be regarded as an image pair with an excessively large attitude angle difference. When screening the image pair, the spherical panoramic image in the image pair will also be screened out.

[0119] Optionally, the attitude angle difference threshold is obtained by the pitch angle difference threshold Δθ′ th and azimuth difference threshold Δφ th Indicates that when any of the pitch angle difference Δθ′ and azimuth angle difference Δφ in the attitude angle difference of the image pair does not meet the attitude angle difference threshold, the image pair is screened out to exclude image pairs that are spatially adjacent but have too large a difference in viewing angle.

[0120] The attitude angle difference threshold obtained is the pitch angle difference threshold Δθ′ th =10°, azimuth difference threshold Δφ th =15°. Based on the attitude angle difference threshold, only the image pairs whose pitch angle difference Δθ′ and azimuth angle difference Δφ both meet the attitude angle difference threshold are retained by screening. The screening formula is as follows:

[0121]

[0122] Among them, S ij The value of is used to represent the image pair (I i ,I j ) whether to enter the subsequent matching process, when S ij When the value is 1, it means that the image pair (I i ,I j ) can enter the subsequent matching process, when S ij When the value is 0, it means that the image pair (I i ,I j ) is screened out and does not enter the subsequent matching process.

[0123] The attitude angle difference threshold based on the spatial constraint model can screen out all image pairs whose attitude angle differences are less than the attitude angle difference threshold. That is, all spherical panoramic images after secondary screening can be obtained based on the screened image pairs for entering the subsequent matching process.

[0124] In practical applications, if the spherical panoramic image pairs in the secondary screening step are reconstructed based on the spherical panoramic images after the primary screening, and the image pairs after the primary screening and deduplication are not directly used, all spherical panoramic images after the secondary screening also need to be deduplicated.

[0125] Step S23: construct a scene graph based on the spherical panoramic image after secondary screening.

[0126] Specifically, a spherical panoramic image after secondary screening is obtained. After the spherical panoramic image after secondary screening is screened based on a dynamic distance threshold and an attitude angle difference threshold, the image pairs constructed are all spatially adjacent, and the differences in attitude angles, especially the pitch angle θ′ and the azimuth angle φ, are small. Therefore, a scene graph can be constructed based on the spherical panoramic images after secondary screening contained in all image pairs.

[0127] Step S30: Divide the scene graph into multiple sub-scene graphs, and prioritize the sub-scene graphs based on a graph order scheduling algorithm.

[0128] It should be noted that the Graph-Order Scheduling (GOS) algorithm is a memory-adaptive graph-order scheduling algorithm used for large-scale image matching scenarios with limited GPU memory resources. Through memory-constrained graph segmentation and sub-scene priority sorting, the global matching task is decomposed into adaptive sub-task streams to balance computational efficiency and resource utilization.

[0129] Step S31: dividing the scene graph into multiple sub-scene graphs based on video memory constraints; wherein the sub-scene graphs include at least two of the filtered spherical panoramic images.

[0130] Specifically, a scene graph is obtained, and based on the scene graph, all secondary filtered spherical panoramic images contained in the scene graph are obtained, each spherical panoramic image is defined as a node, and image pairs constructed and secondary filtered in step S22 are obtained, and each image pair is defined as an edge, that is, the two spherical panoramic images contained in each image pair are the two nodes of the edge.

[0131] Based on the node (spherical panoramic image) and edge (image pair) structure of the scene graph (adjacency graph), a breadth-first search (BFS) strategy is adopted to dynamically expand the sub-scene graph.

[0132] Specifically, starting from the node with the largest degree, that is, from the spherical panoramic image that can match the most image pairs, the adjacent nodes are gradually added to the current sub-scene graph until the number of nodes reaches the upper limit of the video memory capacity N. max (For example, N max = 100). After each sub-scene graph is segmented, its associated edges are removed and the degrees of the remaining nodes are updated. This loop is repeated until the graph is empty. This ensures that the size of each sub-scene graph is strictly limited by the video memory and that the spherical panoramic image within the sub-scene graph has a high adjacency density, which can reduce redundant feature comparisons.

[0133] It should be noted that each sub-scene graph needs to contain at least two filtered spherical panoramic images. Specifically, the filtered spherical panoramic images belong to at least one image pair constructed in step S22 and completed the secondary screening. Therefore, generally each sub-scene graph contains at least two filtered spherical panoramic images, that is, it contains at least one secondary screened image pair.

[0134] Step S32: Calculate the degree of overlap between the current sub-scene graph and other sub-scene graphs in the scene graph; wherein the degree of overlap is defined as the number of shared images between the current sub-scene graph and other sub-scene graphs in the scene graph.

[0135] Specifically, the overlap is defined as the number of shared images between the current subscene graph and other subscene graphs in the scene graph, that is, the subscene graph G is defined as i With G j The overlap of the shared images is the number of overlap(G i ,G j )=|V i ∩V j Based on the subscene graph segmented in step S31, the overlap between the current subscene graph and other subscene graphs in the scene graph is calculated.

[0136] Step S33: Prioritize other sub-scene graphs in the scene graph based on the overlap.

[0137] Based on the result of the overlap calculation in step S32, the priority of processing each sub-scene graph is determined to sort all the sub-scene graphs in the scene graph, and the sub-scene graph with the highest overlap with the spherical panoramic image currently loaded in the video memory is prioritized. By maximizing the video memory data reuse rate and reducing the number of repeated image loading times, the GPU computing efficiency is significantly improved.

[0138] For example, if the sub-scene graph G2 has the largest number of shared images with the processed sub-scene graph G1, and the number of shared images is specifically 30 spherical panoramic images, then the sub-scene graph G2 will be loaded first and only its newly added 70 spherical panoramic images will be uploaded (assuming that the sub-scene graph G2 contains a total of 100 spherical panoramic images).

[0139] Step S40: geometrically correct and extract features from all the filtered spherical panoramic images, and perform feature matching tasks on the spherical panoramic images in the sub-scene graphs according to the priority order of the sub-scene graphs.

[0140] It should be noted that the geometric correction and feature extraction of all the spherical panoramic images after screening are based on all the spherical panoramic images after the secondary screening in step S20, and the obtained spherical panoramic images are first geometrically corrected, and then the feature extraction is performed on the spherical panoramic images after geometric correction to obtain the feature descriptor of the spherical panoramic image, which is used for feature matching of the spherical panoramic image.

[0141] In this application, a feature matching task of a spherical panoramic image is performed by a hash matching algorithm. The hash matching algorithm is specifically a cascade hash feature matching algorithm, and Locality Sensitive Hashing (LSH) is used as a hash function.

[0142] Step S41: performing geometric correction on all the screened spherical panoramic images.

[0143] It is important to note that the projection of the spherical panoramic image onto the cube plane is a key step in the spherical panoramic image processing process. The purpose is to map the pixels of the spherical panoramic image to the corresponding cube plane to reduce geometric distortion and improve the accuracy of feature extraction. The core of this process is to map the pixels in the spherical coordinate system to the local plane coordinate system through mathematical transformation, thereby achieving efficient feature extraction and matching within the local area. Each step of the related process involves coordinate transformation and mathematical calculation.

[0144] In this embodiment, all filtered spherical panoramic images are geometrically corrected. However, for ease of explanation, only one spherical panoramic image is used as an example. The geometric correction method for the remaining spherical panoramic images is identical to that for this spherical panoramic image and will not be further explained. In coordinate conversion and mathematical calculations involving spherical panoramic images, the spherical image in which the pixel is located is the spherical panoramic image.

[0145] Step S411 : obtaining pixel coordinates corresponding to the spherical panoramic images on the cube plane according to all the filtered spherical panoramic images.

[0146] like Figure 3 As shown, one of the spherical panoramic images is obtained from all the filtered spherical panoramic images, and the pixel coordinate of the spherical panoramic image on the cube plane is O c (x,y), pixel coordinate O c (x,y) is usually expressed as an integer, specifically referring to the position coordinates of the upper left corner of the pixel.

[0147] However, if Figure 4 As shown, the actual effective area of ​​the pixel is O c(x, y) is the center of a unit square area. Therefore, in order to accurately represent the center position of the pixel, the coordinates need to be offset. The offset of the X axis and the Y axis are both 0.5, that is, the center position of the pixel coordinate is O' c (x+0.5,y+0.5).

[0148] Step S412: Calculate the orientation vector of the pixel coordinates in the pinhole camera coordinate system and convert it into an orientation vector in the spherical camera coordinate system.

[0149] Specifically, calculate the pixel coordinate O c (x,y) is the orientation vector b in the pinhole coordinate system pinhole The purpose is to convert the pixel coordinates from the image plane to the camera coordinate system into a three-dimensional vector based on the pixel coordinates O in step S411. c (x,y), its orientation vector b in the pinhole camera coordinate system pinhole The calculation formula is as follows:

[0150]

[0151] Where K is the intrinsic parameter matrix of the pinhole camera.

[0152] Get pixel coordinates O c (x,y) is the orientation vector b in the pinhole camera coordinate system pinhole After that, the orientation vector b needs to be rotated by the rotation matrix R pinhole Converted to the spherical camera coordinate system, it becomes the orientation vector b sphere , which facilitates the subsequent finding of the corresponding pixel position on the spherical image. The conversion formula is as follows:

[0153] b sphere =R·b pinhole ;

[0154] The rotation matrix R describes the relative posture relationship between the pinhole camera and the spherical camera. The purpose of this step is to align the orientation vector to the coordinate system of the spherical image for subsequent projection and pixel search.

[0155] Step S413: Acquire the spherical coordinates of the pixel based on the orientation vector of the spherical camera coordinate system.

[0156] Specifically, obtain the spherical camera orientation vector b after the conversion in step S412 sphere , the orientation vector b sphere Project it onto the spherical image, find the corresponding pixel position on the spherical image, and obtain the corresponding spherical coordinates C(u,v).

[0157] Step S414: Obtain pixel values ​​of the spherical camera based on bilinear interpolation, and assign the pixel values ​​to the pixel coordinates.

[0158] Specifically, the pixel value C′(u,v) of the pixel C(u,v) in the spherical camera coordinate system is obtained by bilinear interpolation on the spherical image, and assigned to the pixel O on the cube plane. c (x,y), the bilinear interpolation formula is as follows:

[0159]

[0160] Among them, C′(u i ,v j ) are the coordinates of the four neighboring pixels around the pixel, and are the interpolation weights. The interpolation algorithm is used to smoothly obtain the pixel values ​​of the spherical image to avoid image distortion caused by coordinate discretization.

[0161] In this embodiment, the projection method of the spherical image to the stereo plane can reduce geometric distortion, improve feature extraction accuracy, and optimize the robustness of feature matching. This method also performs well in high-resolution images. Although this method faces challenges in terms of edge discontinuity, computational and storage overhead, it can effectively meet real-time processing requirements when combined with the GPU acceleration and parallelization technology in step S3. However, the cube mapping method has a large computational overhead when processing high-resolution and large-scale data sets, and it is necessary to combine other methods to solve this problem, such as the combination of GPU acceleration in step S3 and the cascade hash algorithm in step S43, which can significantly improve the processing speed and reduce the computational cost through parallel computing while ensuring the accuracy of feature extraction.

[0162] Step S42: extract features from all geometrically corrected spherical panoramic images.

[0163] This embodiment of the present invention uses the SIFT (Scale Invariant Feature Transform) algorithm to extract and describe features from spherical panoramic images. Firstly, the panoramic image undergoes secondary screening and geometric correction, ensuring accurate feature extraction. Secondly, the classic SIFT algorithm is highly scale- and rotation-invariant, robust against viewpoint variations, and has a corresponding high-performance open-source library.

[0164] Specifically, the embodiment of the present invention uses the GPU version of the SPFT algorithm, SIFTGPU, to extract image features and generate descriptors.

[0165] Step S43: performing feature matching tasks of the spherical panoramic images in the sub-scene graphs according to the priority order of the sub-scene graphs.

[0166] It is important to note that the feature matching task of this invention is based on the cascaded hash matching algorithm, which uses hash-based feature matching to convert real-valued feature descriptors into binary hash codes and quickly calculates the similarity scores of feature descriptors through bitwise XOR operations. Compared with a single-layer structure, cascaded hashing uses a three-layer structure to achieve coarse-to-fine feature matching. Due to the data-independence characteristic, cascaded hash feature matching uses Locality Sensitive Hashing (LSH) as the hash function.

[0167] Step S431: Obtain the priority ranking of the sub-scene graph, and perform a feature matching task of the spherical panoramic image in the sub-scene graph based on the priority ranking of the sub-scene graph.

[0168] Specifically, the priority ranking of the sub-scene graphs is obtained. The priority ranking of the sub-scene graphs is based on their overlap, that is, the number of shared images. The sub-scene graph with the highest overlap of the spherical panoramic images loaded in the current GPU is selected, and the feature matching task of all spherical panoramic images in the sub-scene graph is performed based on the obtained priority ranking.

[0169] In actual applications, if the spherical panoramic image is not loaded in the current GPU, the sub-scene graph with the largest number of nodes, that is, the number of spherical panoramic images, is selected. The number of nodes in this sub-scene graph has an upper limit, which is generally the upper limit of the video memory capacity.

[0170] Step S432: Execute feature matching task based on cascade hash matching algorithm.

[0171] In one embodiment of the present invention, the process of cascade hash feature matching is as follows: Figure 5 As shown, it includes four steps: multi-table hash search, hash remapping, K-nearest neighbor search, Euclidean distance calculation, and ratio test. The details are as follows:

[0172] Step S4321: Obtain feature descriptors for extracting features from all the filtered spherical panoramic images, and convert the feature descriptors into short binary hash codes.

[0173] Specifically, feature descriptors obtained by feature extraction of all filtered spherical panoramic images in the current sub-scene are obtained, and all feature descriptors in the spherical panoramic images are mapped into m-bit short binary codes using L groups of hash functions through local sensitive hashing.

[0174] Step S4322: construct a hash lookup table based on the short binary hash code and the locality sensitive hash function, perform a multi-table hash lookup based on the query point in the hash lookup table, and return a first candidate matching point.

[0175] This step is multi-table hash search. For each set of hash functions in L, which are locality-sensitive hash functions, a lookup table is created for each set of hash functions, which contains 2 m buckets. Therefore, for image I i The query point p in the image I j All points in the same bucket will be returned as its candidate matching points.

[0176] In this embodiment, the candidate matching point returned in this step is regarded as the first candidate matching point.

[0177] Step S4323: remap the short binary hash code into a binary hash code, and return a second candidate matching point based on the Hamming distance between the query point and the first candidate matching point.

[0178] This step, known as hash remapping and K-nearest neighbor search, remaps the m-bit rough hash code to an n-bit binary code (n>m) and calculates the Hamming distance between the query point p and the candidate matching points. The candidate matching points are sorted in ascending order based on the Hamming distance, and hash buckets are created using the Hamming distance as the key. All candidate matching points are placed in the corresponding buckets, and the hash buckets are checked sequentially until the number of returned matching points reaches K.

[0179] In this embodiment, the K candidate matching points returned in this step are regarded as second candidate matching points.

[0180] Step S4324: Determine the nearest neighbor and the next nearest neighbor based on the Euclidean distance between the query point and the second candidate matching point, and perform a ratio test on the nearest neighbor using the next nearest neighbor.

[0181] Euclidean distance calculation,Since the accuracy of the Hamming distance of hash coding is lower than the Euclidean distance of the feature descriptor, the Euclidean distances of the first K candidate matching points to the query point are calculated, and the nearest neighbor q1 and the next nearest neighbor q2 are determined.

[0182] Ratio test, if the q1 ratio test passes, dis(q1) is satisfied <dis(q2)*t r , then mark it as the final matching point. r is the ratio test threshold, and dis(*) represents the distance from * to the query point.

[0183] In actual application, the default parameters are used, that is, the number of hash tables L = 6; the number of bits of coarse code and fine code is set to m = 8 and n = 128 respectively; the number of nearest neighbors is set to K = 8. However, in the feature matching based on Approximate Nearest Neighbor (ANN), the ratio threshold t rIt is used to balance the precision and recall rate of the final match. It is usually set to 0.8 in SIFTGPU. However, in the cascade hash matching process, the ratio threshold t r Set to 0.6.

[0184] Step S4325: Establish a feature matching relationship between spherical panoramic images based on the nearest neighbors that pass the ratio test.

[0185] According to the above cascade hash feature matching process, image I can be efficiently established i and I j The feature matching relationship between them.

[0186] Step S50: output the result of spherical panoramic image feature matching.

[0187] Image I i and I j From the same sub-scene, in the establishment of image I i and I j After the feature matching relationship between them is established, it is necessary to establish the feature matching relationship between other images, and then output the corresponding feature matching results based on all the feature matching relationships in the sub-scene.

[0188] Furthermore, after completing the feature matching task of the current sub-scene, it is necessary to complete the feature matching tasks of other sub-scenes according to the established priority order until the feature matching tasks of all sub-scenes in the scene are completed.

[0189] In the present invention, a spherical panoramic image is obtained and preprocessed; the preprocessed spherical panoramic image is screened based on a spatial constraint model, and a scene graph is constructed based on the screened spherical panoramic image; the scene graph is divided into multiple sub-scene graphs, and the sub-scene graphs are prioritized based on a graph sequence scheduling algorithm; geometric correction and feature extraction are performed on all the screened spherical panoramic images, and feature matching tasks of the spherical panoramic images in the sub-scene graphs are performed according to the priority ranking of the sub-scene graphs.

[0190] By using the spatial constraint model to filter the preprocessed spherical panoramic images, a primary screening based on the dynamic distance can retain spatially adjacent spherical panoramic images; a secondary screening based on the attitude angle difference threshold can filter out spherical panoramic images with excessively large attitude angle differences, thereby ensuring the spatial proximity of the images and reducing invalid matching calculations.

[0191] By using a graph-order scheduling algorithm and a sub-scene graph segmentation method based on video memory constraints, the global matching task can be decomposed into a video memory-adaptive sub-task stream. Based on the sub-scene priority sorting, the video memory data reuse rate is maximized and the number of repeated image loadings is reduced. This significantly improves GPU computing efficiency and ensures resource utilization.

[0192] Through geometric correction, geometric distortion can be reduced and the accuracy of feature extraction can be improved, achieving efficient feature extraction and matching; using cascade hash matching, the feature matching relationship between images can be efficiently established, taking into account both the efficiency and accuracy of feature matching.

[0193] Further, if Figure 6 As shown, based on the above-mentioned spherical panoramic image feature matching method, the present invention also provides a spherical panoramic image feature matching system, wherein the spherical panoramic image feature matching system includes:

[0194] An image acquisition module 61 is used to acquire a spherical panoramic image and pre-process the spherical panoramic image;

[0195] A scene construction module 62 is configured to filter the pre-processed spherical panoramic image according to the spatial constraint model and construct a scene graph based on the filtered spherical panoramic image;

[0196] A scene sorting module 63 is configured to divide the scene graph into a plurality of sub-scene graphs and prioritize the sub-scene graphs based on a graph sequence scheduling algorithm;

[0197] A feature matching module 64 is configured to perform geometric correction and feature extraction on all the screened spherical panoramic images, and perform feature matching tasks on the spherical panoramic images in the sub-scene graphs according to the priority order of the sub-scene graphs;

[0198] The result output module 65 is used to output the spherical panoramic image feature matching result.

[0199] Further, if Figure 7 As shown, based on the above-mentioned spherical panoramic image feature matching method and system, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 7 Only some of the components of the terminal are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0200] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal. Furthermore, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as program codes of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a spherical panoramic image feature matching program 40 is stored on the memory 20, and the spherical panoramic image feature matching program 40 can be executed by the processor 10 to implement the method of the present invention.

[0201] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, configured to execute program codes or process data stored in the memory 20, such as executing the method.

[0202] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch screen, etc. The display 30 is used to display information on the terminal and to display a visual user interface.

[0203] In one embodiment, the steps of the above method are implemented when the processor 10 executes the spherical panoramic image feature matching program 40 in the memory 20 .

[0204] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program, and when the program is executed by a processor, the steps of the method described above are implemented.

[0205] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal comprising the element.

[0206] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When the program is executed, it can include the processes in the above-described method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0207] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A spherical panoramic image feature matching method, characterized in that: The spherical panoramic image feature matching method comprises the following steps: Acquire a spherical panoramic image, and pre-process the spherical panoramic image; screening the pre-processed spherical panoramic image based on a spatial constraint model, and constructing a scene graph based on the screened spherical panoramic image; Splitting the scene graph into multiple sub-scene graphs, and prioritizing the sub-scene graphs based on a graph order scheduling algorithm; Performing geometric correction and feature extraction on all the screened spherical panoramic images, and performing feature matching tasks on the spherical panoramic images in the sub-scene graphs according to the priority order of the sub-scene graphs; Output spherical panoramic image feature matching results.

2. The spherical panoramic image feature matching method according to claim 1, characterized in that: The method of screening the pre-processed spherical panoramic image based on the spatial constraint model and constructing a scene graph based on the screened spherical panoramic image includes: Determining a dynamic distance threshold of the spatial constraint model, and screening the pre-processed spherical panoramic image based on the dynamic distance threshold; Determining a posture angle difference threshold of the spatial constraint model, and performing a secondary screening on the spherical panoramic image after the primary screening based on the posture angle difference threshold; Based on the secondary filtered spherical panoramic image, a scene graph is constructed.

3. The spherical panoramic image feature matching method according to claim 2, characterized in that: Determining a dynamic distance threshold of the spatial constraint model and screening the pre-processed spherical panoramic image based on the dynamic distance threshold includes: Determining a dynamic distance threshold of the spatial constraint model, wherein the dynamic distance threshold includes at least a first distance threshold and a second distance threshold; Calculating the Euclidean distance between the current pre-processed spherical panoramic image and other pre-processed spherical panoramic images; Based on the scene density of the location of the current pre-processed spherical panoramic image, a corresponding dynamic distance threshold is selected for screening to obtain a screened spherical panoramic image.

4. The spherical panoramic image feature matching method according to claim 2, characterized in that: Determining a posture angle difference threshold of the spatial constraint model, and performing a secondary screening on the spherical panoramic image after the primary screening based on the posture angle difference threshold, includes: Determining an attitude angle difference threshold of the spatial constraint model; Calculating the difference in attitude angle between the current filtered spherical panoramic image and the other filtered spherical panoramic images; The attitude angle difference is compared with the attitude angle difference threshold, and the spherical panoramic image corresponding to the attitude angle difference that is too large is screened out to obtain a spherical panoramic image after secondary screening.

5. The spherical panoramic image feature matching method according to claim 1, characterized in that: The step of dividing the scene graph into a plurality of sub-scene graphs and prioritizing the sub-scene graphs based on a graph order scheduling algorithm includes: Splitting the scene graph into a plurality of sub-scene graphs based on a video memory constraint; wherein the sub-scene graphs include at least two of the filtered spherical panoramic images; Calculating the degree of overlap between the current subscene graph and other subscene graphs in the scene graph; wherein the degree of overlap is defined as the number of shared images between the current subscene graph and other subscene graphs in the scene graph; Prioritize other sub-scene graphs in the scene graph based on the overlap.

6. The spherical panoramic image feature matching method according to claim 1, characterized in that: The geometric correction of all the filtered spherical panoramic images includes: According to all the filtered spherical panoramic images, obtaining pixel coordinates corresponding to the spherical panoramic images on the cube plane; Calculating the orientation vector of the pixel coordinates in the pinhole camera coordinate system and converting it into the orientation vector of the spherical camera coordinate system; Obtain the spherical coordinates of the pixel based on the orientation vector of the spherical camera coordinate system; A pixel value of the spherical coordinate is obtained based on bilinear interpolation, and the pixel value is assigned to the pixel coordinate.

7. The spherical panoramic image feature matching method according to claim 1, characterized in that: The feature matching task is based on the cascade hash matching algorithm, which specifically includes: Obtaining feature descriptors for extracting features from all the filtered spherical panoramic images, and converting the feature descriptors into short binary hash codes; Constructing a hash lookup table based on the short binary hash code and the locality sensitive hash function, performing a multi-table hash search based on the query point in the hash lookup table, and returning a first candidate matching point; remapping the short binary hash code into a binary hash code, and returning a second candidate matching point based on a Hamming distance between the query point and the first candidate matching point; Determining a nearest neighbor and a second nearest neighbor based on the Euclidean distance between the query point and the second candidate matching point, and performing a ratio test on the nearest neighbor using the second nearest neighbor; Based on the nearest neighbors that pass the ratio test, a feature matching relationship between spherical panoramic images is established.

8. A spherical panoramic image feature matching system, characterized in that: The spherical panoramic image feature matching system includes: An image acquisition module, used for acquiring a spherical panoramic image and preprocessing the spherical panoramic image; A scene construction module is used to filter the pre-processed spherical panoramic images according to the spatial constraint model and construct a scene graph based on the filtered spherical panoramic images; A scene sorting module, configured to divide the scene graph into a plurality of sub-scene graphs and prioritize the sub-scene graphs based on a graph sequence scheduling algorithm; A feature matching module is used to perform geometric correction and feature extraction on all the screened spherical panoramic images, and perform feature matching tasks on the spherical panoramic images in the sub-scene graphs according to the priority order of the sub-scene graphs; The result output module is used to output the spherical panoramic image feature matching results.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a spherical panoramic image feature matching program stored in the memory and runnable on the processor. When the spherical panoramic image feature matching program is executed by the processor, the steps of the spherical panoramic image feature matching method as described in any one of claims 1 to 7 are implemented.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, which stores a spherical panoramic image feature matching program. When the spherical panoramic image feature matching program is executed by a processor, the steps of the spherical panoramic image feature matching method according to any one of claims 1 to 7 are implemented.

Images