Indoor multi-scale scene recognition system and method based on residual fingerprints

Abstract

The invention discloses an indoor multi-scale scene recognition system and method based on residual fingerprints. The indoor multi-scale scene recognition system comprises residual fingerprint construction, node type map construction and an indoor multi-scale scene recognition block. The residual fingerprint construction refers to construction of regional residual features used for representing a region and residual features used for representing a current node image, the node type map construction is used for providing a positioning basis for indoor multi-scale scene recognition, features of the current image are matched with features of the node type map, and a final positioning result is obtained. The method overcomes the defects that a positioning identification method in the prior art can be realized only by arranging wireless equipment, and deployment difficulty and cost need to be increased, and has the advantages of being capable of realizing high-precision positioning in a scene with large light variation, high in positioning precision and strong in robustness.

Description

Technical Field

[0001] This invention relates to the field of positioning technology, and more specifically to an indoor multi-scale scene recognition system and method based on residual fingerprints. Background Technology

[0002] Scene recognition and location of the nearest map node is a crucial step in achieving localization. Traditional indoor positioning methods rely solely on WiFi, Bluetooth, or UWB, requiring a dense deployment of signal transmission base stations to match signal strength for scene recognition. The high cost of these base stations limits their widespread adoption.

[0003] Currently, there have been some achievements in indoor scene recognition. For example, the published patent, application number 202111627216.7, entitled "A Multi-Scale Scene Recognition Method for Underground Parking Lots Based on Anchor Point Images," discloses a scene recognition method using anchor point images. However, the anchor point images are representative images selected within a region. Due to limitations in the selection method, the anchor point images cannot fully represent the region.

[0004] For example, patent CN110360999A, granted on March 26, 2018, entitled "Indoor Positioning Method, Indoor Positioning System, and Computer-Readable Medium," discloses a positioning method that combines computer vision and wireless technology. This method requires the deployment of wireless devices, increasing deployment difficulty and cost.

[0005] Therefore, there is an urgent need for a solution to the above problems. Summary of the Invention

[0006] The primary objective of this invention is to overcome the shortcomings of the aforementioned background technology and to propose an indoor multi-scale scene recognition system based on residual fingerprints.

[0007] The first objective of this invention is achieved through the following technical solution: an indoor multi-scale scene recognition system based on residual fingerprints, comprising residual fingerprint construction, node-based map construction, and an indoor multi-scale scene recognition block; wherein the residual fingerprint construction refers to constructing regional residual features to characterize the region and node residual features to characterize the current node image, the node-based map construction is used to provide a positioning basis for indoor multi-scale scene recognition, and the indoor multi-scale scene recognition module is used to integrate the specific features to obtain the specific positioning point of the target.

[0008] The second objective of this invention is to overcome the shortcomings of the aforementioned background technology and to propose a method for an indoor multi-scale scene recognition system based on residual fingerprints.

[0009] The second objective of this invention is achieved through the following technical solution: a method for an indoor multi-scale scene recognition system based on residual fingerprints, characterized in that: the residual fingerprint includes regional residual features and node residual features, both of which are extracted from a front-view image.

[0010] A method for constructing residual fingerprints based on regional residual features and node residual features, comprising the following steps;

[0011] ① Load the front view image of a node into ResNet and extract the specific residual layer of ResNet to obtain the residual features of the image, i.e., the node residual features;

[0012] ② Cluster the node residual features of the region using the K-means algorithm and select the node residual features that can represent the unique features of the region;

[0013] ③ Average the selected node residual features to obtain the regional residual features used to characterize the region;

[0014] ④ Use a tree structure to construct a residual fingerprint from the regional residual features in step 3 and the node residual features in step 1.

[0015] In the above technical solution: the residual fingerprint obtained in step ④ is used to construct a node-based map.

[0016] Node-based map construction includes the following steps: at regular intervals, a front view image is collected, and the residual fingerprint of the node is extracted to construct a node, while providing data for subsequent positioning.

[0017] In the above technical solution: indoor scene recognition is performed by matching residual fingerprints at multiple scales. The indoor multi-scale scene recognition method includes the following steps: matching residual fingerprints in step ④ using hierarchical matching.

[0018] In the above technical solution, indoor multi-scale scene recognition includes the following specific steps: First, by matching residual fingerprints, the area where the node to be located is located is located. Then, the residual features of the nodes in the area are matched one by one to locate the map node closest to the node to be located, thus obtaining the specific location point of the target.

[0019] The present invention has the following advantages: 1. The present invention constructs residual fingerprints by extracting residual layer features in ResNet, and constructs a node map based on residual fingerprints, and finally realizes multi-scale scene recognition.

[0020] 2. This invention includes regional residual features for characterizing regions and node residual features for representing nodes. A map containing residual fingerprints is then constructed, and based on this, multi-scale scene recognition is achieved, improving positioning accuracy and efficiency.

[0021] 3. The indoor multi-scale scene recognition in this invention achieves coarse-to-fine localization by hierarchically matching residual fingerprints. First, by matching regional residual features, the region where the node to be located is located is identified. Then, the residual features of nodes in that region are matched one by one to locate the map node closest to the node to be located, thereby achieving indoor scene recognition of the target.

[0022] 4. Compared to the existing patent application number 202111627216.7, entitled "A Multi-Scale Scene Recognition Method for Underground Parking Lots Based on Anchor Point Images", which discloses a scene recognition method using anchor point images, where the anchor point images are representative images selected within the region, the anchor point images are limited by the selection method, resulting in the anchor point images not being able to fully represent the region. The present invention uses the K-means algorithm to cluster the node residual features, thereby obtaining regional features with stronger representation capabilities.

[0023] 5. Compared to, for example, the already granted patent CN110360999A, granted on March 26, 2018, entitled "Indoor Positioning Method, Indoor Positioning System, and Computer-Readable Medium," which discloses a positioning method utilizing a combination of computer vision and wireless technology, this method requires the deployment of wireless equipment, increasing deployment difficulty and cost. This invention achieves precise positioning solely through map construction, eliminating the need for additional equipment deployment, resulting in significantly lower costs and easier implementation.

[0024] 6. This invention extracts residual fingerprints from images and performs hierarchical matching of residual fingerprints, enabling high-precision positioning even in scenarios with large changes in indoor lighting. This invention features high positioning accuracy and strong robustness. Attached Figure Description

[0025] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0026] The technical solutions in the embodiments of the present invention will be clearly and completely described below. All other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present invention.

[0027] Reference Figure 1As shown: An indoor multi-scale scene recognition system based on residual fingerprints includes residual fingerprint construction, node-based map construction, and indoor multi-scale scene recognition blocks. The residual fingerprint construction refers to constructing regional residual features to represent the region and node residual features to represent the current node image. The node-based map construction is used to provide positioning basis for indoor multi-scale scene recognition. The indoor multi-scale scene recognition module is used to integrate the specific features to obtain the specific positioning point of the target.

[0028] The residual fingerprint includes region residual features and node residual features, both of which are extracted from the front view image.

[0029] A method for constructing residual fingerprints based on regional residual features and node residual features, comprising the following steps;

[0030] ① By imaging the region residual features and node residual features through the front view image, the image is loaded into ResNet, and specific residual layers of ResNet are extracted to obtain the residual features of the image.

[0031] ② Cluster the node residual features of the region using the K-means algorithm, and select residual features that can represent the unique characteristics of the region;

[0032] ③ Average the selected node residual features to obtain the regional residual features used to characterize the region;

[0033] ④ Use a tree structure to construct a residual fingerprint from the regional residual features in step 3 and the node residual features in step 1.

[0034] The residual fingerprints obtained in step ④ are used to construct a node-based map.

[0035] Node-based map construction includes the following steps: at regular intervals, a front view image is collected, and the residual fingerprint of the node is extracted to construct a node, while providing data for subsequent positioning.

[0036] Indoor scene recognition is performed by matching residual fingerprints at multiple scales. The indoor multi-scale scene recognition method includes the following steps: matching residual fingerprints in step ④ using hierarchical matching.

[0037] Indoor multi-scale scene recognition includes the following specific steps: First, by matching residual region features, the region where the node to be located is located is located. Then, the residual features of the nodes in the region are matched one by one to locate the map node closest to the node to be located, thus obtaining the specific location point of the target.

[0038] Example: An indoor multi-scale scene recognition system based on residual fingerprints, which includes the following specific steps: ① Residual fingerprint construction method;

[0039] The residual fingerprint is constructed from residual region features and node residual features; where residual region features are used to characterize the region, and node residual features are used to characterize the current node. By extracting specific residual layer features from ResNet and vectorizing these residual layer features, the residual feature F is obtained.

[0040] F = [α1, α2, ..., α N (1)

[0041] In the formula, α i Let N be the i-th element in the vector, and N be the length of the vector.

[0042] Different regions possess different characteristics, such as letter markings and facilities. To extract images with unique features from each region, K-means clustering of the node residual features can effectively represent the region's residual characteristics. Then, to obtain the regional residual features used to represent the region, the obtained node residual features are mean-sampled to obtain the regional residual feature R:

[0043]

[0044] Where m is the number of images in the region, α ij The j-th element represents the residual feature of the i-th node.

[0045] Finally, the residual features of the region and the residual features of the nodes are combined using a tree structure to obtain the residual fingerprint P:

[0046] P i ={R i F i} (3)

[0047] ② Residual fingerprint node-based map construction method;

[0048] The residual fingerprint node map construction method involves acquiring a front view image at regular intervals and extracting the residual fingerprint P. This process is repeated until the indoor environment is sufficiently captured, resulting in a map m that characterizes the indoor environment.

[0049] m = {P1, P2, P3, ..., P} N} (4)

[0050] In the formula, N is the number of nodes in the map.

[0051] ③ An indoor multi-scale recognition method based on residual fingerprints;

[0052] The indoor multi-scale identification method based on residual fingerprints adopts a coarse-to-fine localization strategy, which includes two steps: initial localization based on regional residual features and fine localization based on node residual features.

[0053] ④ Initial localization based on regional residual characteristics;

[0054] Initial localization based on regional residual features involves comparing the residual features F of the node to be located with the i-th region R in the map. i To perform matching, the region with the smallest cosine distance d between two vectors is the region to which the vector belongs.

[0055] d = 1 - Cos(F, R) i (5)

[0056] k′=argmin[d1, d2, d3,...d N (6)

[0057] The above steps yield the region where the node to be located is located. The resulting region contains t candidate map nodes, as shown below:

[0058] D k′ = [P1, P2, ..., P t (7)

[0059] ⑤ Fine localization based on node residual features

[0060] Then, by calculating the cosine distance between the residual features of the candidate map nodes and the node to be located, the map node with the smallest distance is the obtained localization result.

[0061] d = 1 - Cos(F) mapi ,F) (8)

[0062] In the formula: F mapi Let F be the residual feature of the i-th candidate map node, and F be the residual feature of the node to be located. The map node with the smallest cosine distance to the node to be located is the final localization result.

[0063] result = argmin[d1, d2, d3, ... d t (9)

[0064] The parts not described in detail above are all existing technologies.

Claims

1. An indoor multi-scale scene recognition system based on residual fingerprints, characterized in that: It includes residual fingerprint construction, node-based map construction, and indoor multi-scale scene recognition blocks; the residual fingerprint construction refers to constructing regional residual features to characterize the region and node residual features to characterize the current node image; the node-based map construction is used to provide positioning basis for indoor multi-scale scene recognition; and the indoor multi-scale scene recognition is used to integrate the specific features to obtain the specific positioning point of the target.

2. A method for an indoor multi-scale scene recognition system based on residual fingerprints as described in claim 1, characterized in that: The residual fingerprint includes region residual features and node residual features, both of which are extracted from the front view image. A method for constructing residual fingerprints based on regional residual features and node residual features, comprising the following steps; ① Load the front view image of a node into ResNet and extract the specific residual layer of ResNet to obtain the node residual features of the image; ② Cluster the node residual features of the region using the K-means algorithm and select residual features that can represent the unique characteristics of the region; ③ Average the selected node residual features to obtain the regional residual features used to characterize the region; ④ Use a tree structure to construct a residual fingerprint from the regional residual features in step ③ and the node residual features in step ①.

3. The indoor multi-scale scene recognition method based on residual fingerprints as described in claim 2, characterized in that: The residual fingerprints obtained in step ④ are used to construct a node-based map. Node-based map construction includes the following steps: at regular intervals, a front view image is collected, and the residual fingerprint of the node is extracted to construct a node, while providing data for subsequent positioning.

4. The indoor multi-scale scene recognition method based on residual fingerprints as described in claim 3, characterized in that: Indoor multi-scale scene recognition is performed based on the data provided by subsequent positioning. Indoor scene recognition is performed by matching residual fingerprints at multiple scales: the residual fingerprints in step ④ are matched hierarchically.

5. The indoor multi-scale scene recognition method based on residual fingerprints according to claim 4, characterized in that: Indoor multi-scale scene recognition includes the following specific steps: First, by matching residual fingerprints, the area where the node to be located is located is located. Then, the residual features of the nodes in the area are matched one by one to locate the map node closest to the node to be located, thus obtaining the specific location point of the target.

6. The indoor multi-scale scene recognition method based on residual fingerprints according to claim 5, characterized in that: Residual fingerprint construction method; The residual fingerprint is constructed from residual region features and node residual features; where residual region features are used to characterize the region, and node residual features are used to characterize the current node. By extracting specific residual layer features from ResNet and vectorizing these residual layer features, the residual feature F is obtained. F＝[α1,α2,…,α N ] (1) In the formula, α i Let N be the i-th element in the vector, and N be the length of the vector. Different regions possess different characteristics, such as letter identifiers and facilities. To extract images with unique features from each region, K-means clustering of the node residual features can effectively characterize the region's residual features. Then, to obtain the regional residual features used to characterize the region, the obtained node residual features are mean-sampled to obtain the regional residual feature R: Where m is the number of images in the region, α ij The j-th element represents the residual feature of the i-th node. Finally, the residual features of the region and the residual features of the nodes are combined using a tree structure to obtain the residual fingerprint P: P i ={R i ,F i } (3) 7. The indoor multi-scale scene recognition method based on residual fingerprints according to claim 6, characterized in that: A method for constructing node-based maps using residual fingerprints; The residual fingerprint node map construction method involves acquiring a front view image at regular intervals and extracting the residual fingerprint P. This process is repeated until the indoor environment is sufficiently captured, resulting in a map m that characterizes the indoor environment. m={P1,P2,P3, …,P N } (4) In the formula, N is the number of nodes in the map.

8. The indoor multi-scale scene recognition method based on residual fingerprints according to claim 6, characterized in that: Indoor multi-scale recognition method based on residual fingerprints; The indoor multi-scale identification method based on residual fingerprints adopts a coarse-to-fine localization strategy, which includes two steps: initial localization based on regional residual features and fine localization based on node residual features.

9. The indoor multi-scale scene recognition method based on residual fingerprints according to claim 8, characterized in that: Initial localization based on regional residual characteristics; Initial localization based on regional residual features involves comparing the residual features F of the node to be located with the i-th region R in the map. i To perform matching, the region with the smallest cosine distance d between two vectors is the region to which the vector belongs. d=1-Cos(F,R i ) (5) k′=argmin[d1,d2,d3,…d N (6) The above steps yield the region where the node to be located is located. The resulting region contains t candidate map nodes, as shown below: D k′ =[P1,P2,…,P t ] (7) 10. The indoor multi-scale scene recognition method based on residual fingerprints according to claim 8, characterized in that: Fine-grained localization based on node residual features; Then, by calculating the cosine distance between the residual features of the candidate map nodes and the node to be located, the map node with the smallest distance is the obtained localization result. d=1-Cos(F mapi ,F) (8) In the formula: F mapi Let F be the residual feature of the i-th candidate map node, and F be the residual feature of the node to be located. The map node with the smallest cosine distance to the node to be located is the final localization result. result=argmin[d1,d2,d3,…d t (9)

Images