Multi-scale vector image retrieval and positioning method, system and medium for indoor scenes

Abstract

The present invention discloses a method, system and medium for multi-scale vector image retrieval and positioning of indoor scenes. The method comprises the following steps: obtaining an indoor panoramic image by taking a panoramic shot of the surrounding environment at a central position indoors; performing slicing and cubemap spatial transformations on the indoor panoramic image; extracting slice image features from the results obtained by the slicing and cubemap spatial transformations; extracting target image features from a target image taken at any position to be positioned, and determining the target position corresponding to the target image to be positioned based on the correlation between the target image features and the slice image features. In response to the high data quality requirements and high computing resource requirements of existing registration-based positioning methods in indoor scenes, which are difficult to implement in engineering applications, the present invention can perform coarse positioning based on cube slices of panoramic images and fine positioning based on slices of panoramic images, thereby achieving high-speed image positioning in a real-time environment.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method, system and medium for retrieval and positioning of multi-scale vector images of indoor scenes. Background Art

[0002] Image-based positioning algorithms, as an interdisciplinary technology that integrates computer vision, machine learning, multi-view geometry, image retrieval, and other scientific research fields, have broad application prospects and enormous research value in areas such as robot navigation and positioning, augmented reality, 3D reconstruction, and landmark recognition. Image-based positioning technology is also an indispensable component of high-level image processing tasks. Currently, positioning mainly adopts the following two methods: Point cloud registration: Point cloud registration (Point Cloud Registration), also known as point cloud stitching or point cloud registration, for point clouds with overlapping information in two frames, by solving the transformation matrix (rotation matrix R and translation matrix T), the overlapping point clouds are transformed into the same unified coordinate system. However, due to the difficulty in obtaining a complete 3D point cloud model of the scene and the target to be registered, and the large amount of point cloud data, the computational overhead of registration, this makes point cloud registration-based positioning methods difficult to apply in practical engineering applications. Image registration: Image registration is the process of matching and overlaying two or more images acquired at different times, using different sensors (imaging devices), or under different conditions (such as weather, illumination, camera position, and angle). It has been widely used in fields such as remote sensing data analysis, computer vision, and image processing. The process of image registration technology is as follows: first, feature points are extracted from the two images; matching feature point pairs are found through similarity measurements; then, image space coordinate transformation parameters are derived from these matching feature point pairs; and finally, image registration is performed based on these coordinate transformation parameters. However, in practical applications, this consumes significant computational resources, making it difficult to implement in practice. However, the computational resources required for both point cloud and image registration make their practical application difficult, and the high accuracy requirements further limit their general application. Summary of the Invention

[0003] The technical problem to be solved by the present invention is as follows: In view of the fact that the existing registration-based high requirements for data quality and high requirements for computing resources are difficult to actually implement in engineering applications, the present invention provides a positioning method based on image retrieval, which performs coarse positioning based on cube slices of panoramic images and fine positioning based on slices of panoramic images, and can achieve high-speed image positioning in a real-time environment.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0005] A multi-scale vector image retrieval and positioning method for indoor scenes, comprising:

[0006] S101, obtaining an indoor panoramic image by panoramically photographing the surrounding environment at a central position indoors;

[0007] S102, performing slicing and cubemap spatial transformation on the indoor panoramic image;

[0008] S103, extracting slice image features from the results of spatial transformation of the slice and cube map respectively;

[0009] S104 , extracting target image features from a target image captured at any position to be positioned, and determining a target position corresponding to the target image to be positioned according to a correlation between the target image features and the slice image features.

[0010] Optionally, the indoor panoramic image in step S101 is a panoramic image P with a size of H×W obtained by shooting using a cylindrical projection model.

[0011] Optionally, step S102 includes: slicing the panoramic image P with a size of H×W using horizontal and vertical axes to obtain n image slices P1 to P1 with a size of h1×w1. n ; Perform cubemap transformation on the panoramic image P of size H×W, and transform the 360° panoramic image P into 6 single-view images C1~C6 of size h2×w2 according to the six sides of front, back, left, right, top and bottom.

[0012] Optionally, step S103 includes: respectively slicing n image slices P1 to P1 with a size of h1×w1 n Image feature extraction is performed to obtain n image slice feature vectors of size 1×C; image feature extraction is performed on 6 single-view images C1 to C6 of size h2×w2 respectively to obtain 6 single-view feature vectors of size 6.

[0013] Optionally, the target image in step S104 is a target image I captured by a monocular camera at any desired positioning position.

[0014] Optionally, extracting target image features in step S104 refers to performing image feature extraction on the target image I to obtain a feature vector with a size of 1×C.

[0015] Optionally, determining the target position corresponding to the target image to be located according to the correlation between the target image feature and the slice image feature in step S104 includes:

[0016] S201, for a feature vector of the target image I with a size of 1×C, calculate the distance between it and n image slice feature vectors with a size of 1×C, and obtain n slice feature distances d 1,1 ~d 1,n ; For a feature vector of size 1×C of the target image I, the distances between it and the six single-view feature vectors of size 6 are calculated to obtain the six single-view feature distances d 2,1 ~d 2,6 ;

[0017] S202, at n slice feature distances d 1,1 ~d 1,n Find the minimum slice feature distance d p , determine the minimum slice feature distance d p The corresponding image slice P p ; In the 6 single view feature distances d 2,1 ~d 2,6 Find the minimum single-view feature distance d q , determine the minimum single-view feature distance d q The corresponding single-view image C q ;

[0018] S203, calculate the found image slice P p , single-view image C q The distance between them is ds;

[0019] S204: If the distance ds is less than the preset threshold s, it is determined that the target position corresponding to the target image to be located belongs to the image slice P. p If it does not hold, it is determined that the target position corresponding to the target image to be located does not belong to the image slice P p If the corresponding image slice cannot be found in the end, it is determined that the target position corresponding to the target image to be located has failed to be located.

[0020] Optionally, the distances in steps S201 and S203 are both Euclidean distances.

[0021] In addition, the present invention also provides an indoor scene multi-scale vector image retrieval and positioning system, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the indoor scene multi-scale vector image retrieval and positioning method.

[0022] In addition, the present invention also provides a computer-readable storage medium, in which a computer program is stored. The computer program is used to be programmed or configured by a microprocessor to execute the indoor scene multi-scale vector image retrieval and positioning method.

[0023] Compared with the existing technology, the present invention has the following main advantages: the present invention includes obtaining an indoor panoramic image obtained by taking a panoramic shot of the surrounding environment at a central position indoors; performing slicing and cubemap spatial transformation on the indoor panoramic image; extracting slice image features from the results obtained by the slicing and cubemap spatial transformation; extracting target image features from a target image taken at any position to be positioned, and determining the target position corresponding to the target image to be positioned based on the correlation between the target image features and the slice image features. In response to the high data quality requirements and high computing resource requirements of existing registration-based positioning methods in indoor scenes, which make it difficult to actually implement engineering applications, the present invention can perform coarse positioning based on cube slices of panoramic images and fine positioning based on slices of panoramic images, and can achieve high-speed image positioning in a real-time environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the basic process of the method of the embodiment of the present invention.

[0025] Figure 2 Schematic diagram of input according to an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of outputting positioning results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] like Figure 1 As shown, the indoor scene multi-scale vector image retrieval and positioning method of this embodiment includes:

[0028] S101, obtain an indoor panoramic image obtained by taking a panoramic shot of the surrounding environment at the middle position of the room, such as Figure 2 As shown;

[0029] S102, performing slicing and cubemap spatial transformation on the indoor panoramic image;

[0030] S103, extracting slice image features from the results of spatial transformation of the slice and cube map respectively;

[0031] S104, extracting target image features from any target image taken at the position to be positioned, and determining the target position corresponding to the target image to be positioned according to the correlation between the target image features and the slice image features, such as Figure 3 shown.

[0032] The indoor panoramic image in step S101 of this embodiment is a panoramic image P of size H×W obtained by shooting using a cylindrical projection model. When shooting a panoramic image of the surrounding environment at the center of the indoor space, try to shoot the panoramic image of the surrounding environment at the center of the scene space, and use the cylindrical projection model to obtain the panoramic image P of the indoor space of size H×W.

[0033] In this embodiment, step S102 includes: slicing the panoramic image P with a size of H×W using the horizontal and vertical axes as the slicing axes to obtain n image slices P1 to P1 with a size of h1×w1. n ; Perform cubemap transformation on the panoramic image P of size H×W, and transform the 360° panoramic image P into 6 single-view images C1~C6 of size h2×w2 according to the six sides of front, back, left, right, top and bottom.

[0034] In this embodiment, step S103 includes: respectively performing a multi-step process on n image slices P1 to P1 with a size of h1×w1. n Perform image feature extraction to obtain n image slice feature vectors of size 1×C; perform image feature extraction on 6 single-view images C1~C6 of size h2×w2 respectively, and obtain 6 single-view feature vectors of size 6. It should be noted that for n image slices P1~P n Image feature extraction can be performed using a desired convolutional neural network as needed. For example, as an optional implementation, the steps for image feature extraction in this embodiment include: using ResNet to perform basic feature extraction on an input image of size H×W, and obtaining a feature map of size H / 8×W / 8*512 through three convolutional layers. Then, using the feature map of size H / 8×W / 8*512 as input, two convolutional layers and a channel average layer are used to obtain a feature map of size H / 16×W / 16*1. Then, by constructing convolutional layers and performing training calculations, using the feature map of size H / 16×W / 16*1 as input, a feature vector of size 1×HW / 256 is obtained as the retrieval feature.

[0035] In this embodiment, the target image in step S104 is a target image I captured by a monocular camera at any desired positioning position. The target image captured at any position to be positioned is captured monocularly using any camera and is recorded as I for use in step S104.

[0036] In this embodiment, extracting target image features in step S104 refers to performing image feature extraction on the target image I to obtain a feature vector with a size of 1×C.

[0037] In this embodiment, determining the target position corresponding to the target image to be located according to the correlation between the target image feature and the slice image feature in step S104 includes:

[0038] S201, for a feature vector of the target image I with a size of 1×C, calculate the distance between it and n image slice feature vectors with a size of 1×C, and obtain n slice feature distances d 1,1 ~d 1,n; For a feature vector of size 1×C of the target image I, the distances between it and the six single-view feature vectors of size 6 are calculated to obtain the six single-view feature distances d 2,1 ~d 2,6 ;

[0039] S202, at n slice feature distances d 1,1 ~d 1,n Find the minimum slice feature distance d p , determine the minimum slice feature distance d p The corresponding image slice P p ; In the 6 single view feature distances d 2,1 ~d 2,6 Find the minimum single-view feature distance d q , determine the minimum single-view feature distance d q The corresponding single-view image C q ;

[0040] S203, calculate the found image slice P p , single-view image C q The distance between them is ds;

[0041] S204: If the distance ds is less than the preset threshold s, it is determined that the target position corresponding to the target image to be located belongs to the image slice P. p If it does not hold, it is determined that the target position corresponding to the target image to be located does not belong to the image slice P p If the image slice to which it belongs cannot be found, it is determined that the target position corresponding to the target image to be located has failed to be located. In this embodiment, the distances in steps S201 and S203 are both Euclidean distances. In addition, other geometric distances can be used to replace the Euclidean distance as needed.

[0042] In summary, the method includes obtaining an indoor panoramic image obtained by taking a panoramic shot of the surrounding environment at a central position indoors; performing slicing and cubemap spatial transformations on the indoor panoramic image; extracting slice image features from the results obtained by the slicing and cubemap spatial transformations; extracting target image features from a target image taken at any position to be positioned, and determining the target position corresponding to the target image to be positioned based on the correlation between the target image features and the slice image features. The present invention addresses the high data quality requirements and high computing resource requirements of existing registration-based positioning methods in indoor scenes, which are difficult to implement in engineering applications. The present invention can perform coarse positioning based on cube slices of panoramic images and fine positioning based on slices of panoramic images, and can achieve high-speed image positioning in a real-time environment.

[0043] Furthermore, this embodiment provides an indoor scene multi-scale vector image retrieval and positioning system for an image positioning system for indoor scenes, comprising an interconnected microprocessor and memory, wherein the microprocessor is programmed or configured to execute the indoor scene multi-scale vector image retrieval and positioning method. Furthermore, this embodiment provides a computer-readable storage medium storing a computer program for being programmed or configured by the microprocessor to execute the indoor scene multi-scale vector image retrieval and positioning method.

[0044] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0045] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A multi-scale vector image retrieval and positioning method for indoor scenes, characterized by: include: S101, obtaining an indoor panoramic image by panoramically photographing the surrounding environment at a central position indoors; S102, performing slicing and cubemap spatial transformation on the indoor panoramic image; S103, extracting slice image features from the results of spatial transformation of the slice and cube map respectively; S104, extracting target image features from a target image captured at any position to be located, and determining a target position corresponding to the target image to be located based on a correlation between the target image features and the slice image features; In step S104, determining the target position corresponding to the target image to be located according to the correlation between the target image feature and the slice image feature includes: S201, for the target image I 1×C feature vector, respectively calculated with n The distance between the feature vectors of image slices of size 1×C is obtained n Slice feature distance d 1,1 ~ d 1,n ; For the target image I The distance between the 1×C feature vector and the 6 single-view feature vectors of size 6 is calculated to obtain 6 single-view feature distances. d 2,1 ~ d 2,6 ; S202, in n Slice feature distance d 1,1 ~ d 1,n Find the minimum slice feature distance d p , determine the minimum slice feature distance d p The corresponding image slice P p ; At 6 single-view feature distances d 2,1 ~ d 2,6 Find the minimum single-view feature distance d q , determine the minimum single-view feature distance d q The corresponding single-view image C q ; S203, calculate the found image slices P p , single-view image C q The distance between ds ; S204, if the distance ds If the value is less than the preset threshold s, it is determined that the target position corresponding to the target image to be located belongs to the image slice. P p If it does not hold, it is determined that the target position corresponding to the target image to be located does not belong to the image slice P p ; If the corresponding image slice cannot be found in the end, it is determined that the target position corresponding to the target image to be located has failed to be located.

2. The method for indoor scene multi-scale vector image retrieval and positioning according to claim 1, characterized in that: The indoor panoramic image in step S101 is a panoramic image of size H×W obtained by shooting with a cylindrical projection model. P .

3. The method for indoor scene multi-scale vector image retrieval and positioning according to claim 2, characterized in that: Step S102 includes: using the horizontal and vertical axes as slice axes, slicing the panoramic image with a size of H×W P Slice and obtain n The size is h 1× w 1 image slice P 1~ P n ; Panoramic image of size H×W P Perform cubemap transformation to transform the 360° panoramic image P According to the front, back, left, right, top and bottom six sides are transformed into 6 sizes h 2× w 2 single-view image C 1~ C 6.

4. The method for indoor scene multi-scale vector image retrieval and positioning according to claim 3, characterized in that: Step S103 includes: h 1× w 1 image slice P 1~ P n Perform image feature extraction to obtain n image slice feature vectors of size 1×C; perform image feature extraction on 6 image slices of size h 2× w 2 single-view image C 1~ C 6 performs image feature extraction and obtains 6 single-view feature vectors of size 6.

5. The method for indoor scene multi-scale vector image retrieval and positioning according to claim 4, characterized in that: The target image in step S104 is the target image captured by a monocular camera at any desired location. I .

6. The method for indoor scene multi-scale vector image retrieval and positioning according to claim 5, characterized in that: Step S104 is to extract the target image features for the target image. I Perform image feature extraction to obtain a feature vector of size 1×C.

7. The method for indoor scene multi-scale vector image retrieval and positioning according to claim 6, characterized in that: The distances in steps S201 and S203 are both Euclidean distances.

8. An indoor scene multi-scale vector image retrieval and positioning image positioning system for indoor scenes, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the indoor scene multi-scale vector image retrieval and positioning method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, wherein: The computer program is used to be programmed or configured by a microprocessor to execute the indoor scene multi-scale vector image retrieval and positioning method according to any one of claims 1 to 7.

Images