Indoor visual positioning system and method based on fuzzy image

Abstract

The invention discloses an indoor visual positioning system and method based on fuzzy images, which relates to the field of image processing technology. A plurality of reference points are set, and a plurality of images are collected as database images based on the reference point positions; the features of the database images are extracted and saved as database image feature information using the SuperPoint algorithm and the NetVLAD algorithm; the pose labels of the database images are obtained based on the reference points and stored corresponding to the database images to obtain an offline database; a query image is determined, and whether the query image is a fuzzy image is determined based on a fuzziness degree model BDM, and feature extraction of the restored image is performed based on the SuperPoint algorithm to obtain restored image feature information, and the database, the query image, and all feature extraction and matching information are finally positioned using the epipolar geometry five-point method. The invention gives full play to the advantages of the NetVLAD algorithm being fast and having high retrieval efficiency, and the SuperPoint algorithm being accurate and having strong robustness, and improves the overall performance of the indoor positioning algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an indoor visual positioning system and method based on fuzzy images. Background Art

[0002] With the rapid development of intelligent network devices and Internet of Things technology, people's demand for indoor positioning accuracy and efficiency is increasing. Indoor positioning technology has broad application prospects in navigation and location services, shopping mall sales, industrial manufacturing and other fields. At present, due to the complexity of the indoor environment, images may be distorted by various factors, the most common of which is image blur. Image blur is caused by the movement of the camera or object during shooting, which makes the details in the image unclear. Blurred images will have a negative impact on the feature extraction and matching process, the pose estimation process, and reduce positioning accuracy. Summary of the invention

[0003] The purpose of the present invention is to provide an indoor visual positioning system and method based on fuzzy images to solve the shortcomings of the background technology.

[0004] In order to achieve the above object, the present invention provides the following technical solution: an indoor visual positioning method based on fuzzy images, the steps of setting the offline stage include:

[0005] Setting a plurality of reference points, and acquiring a plurality of images as database images based on the positions of the reference points;

[0006] Use SuperPoint algorithm and NetVLAD algorithm to extract the features of database images as database image feature information and save them;

[0007] The pose labels of the database images are obtained based on the reference points and stored corresponding to the database images to obtain an offline database;

[0008] The steps to set up the online stage include:

[0009] Determine a query image, and judge whether the query image is a blurred image based on a blur degree model BDM. If the query image is a blurred image, use an image restoration network to restore the blurred image to obtain a restored image.

[0010] The restored image feature information is obtained by extracting features based on the SuperPoint algorithm, and the feature information is retrieved by using a retrieval algorithm based on the offline database to perform a nearest neighbor retrieval.

[0011] The 2D-2D matching information between the query image and the database image is established by feature matching using the SuperGlue algorithm, image pairs and SuperPoint feature information;

[0012] The database, query image and all feature extraction and matching information are finally positioned using the epipolar geometry five-point method.

[0013] In a preferred embodiment, the step of setting a plurality of reference points and acquiring a plurality of images as database images based on the positions of the reference points includes:

[0014] Acquire indoor space information to be captured by an image, wherein the indoor space information includes an indoor space structure and a corresponding space size;

[0015] A plurality of reference points are set in the indoor space, and corresponding reference spaces are divided according to the reference points to obtain a plurality of reference areas;

[0016] A plurality of images are acquired in the reference area based on the reference point positions as database images.

[0017] In a preferred embodiment, the step of acquiring the pose label of the database image based on the reference point and storing the corresponding database image to obtain the offline database includes:

[0018] A coordinate system is established in the reference area using the reference point as the origin, and the position and pose of the database image in the reference area is obtained according to the reference point;

[0019] Bind the pose to the corresponding image, and use the pose as the pose label of the image;

[0020] The pose labels, the corresponding images and the corresponding database image feature information are stored as an offline database.

[0021] In a preferred embodiment, the step of storing the pose label, the corresponding image and the corresponding database image feature information as an offline database includes:

[0022] A corresponding number of storage bins are set corresponding to the number of reference areas, and an update bin is set corresponding to the storage bins;

[0023] Obtain the positional relationship between the reference areas, connect the storage bins based on the positional relationship and establish a reference area model of the storage bins for positional arrangement, and spatially associate the storage bins corresponding to the reference areas based on the line of sight channels and reference points;

[0024] Extracting image features from all reference areas, and binding information between the image features and the corresponding reference areas as the first matching area;

[0025] Dividing the space in the storage bin with spatial association to obtain a plurality of association spaces, and binding the image features with the association spaces as the second matching area;

[0026] The image features in the first matching area and the second matching area are used as matching labels, and the first matching area, the second matching area and the corresponding matching labels are used as database image feature information;

[0027] The pose labels, the corresponding images and the corresponding database image feature information are stored as an offline database.

[0028] In a preferred embodiment, the step of determining the query image, judging whether the query image is a blurred image based on the blur degree model BDM, and if the query image is a blurred image, restoring the blurred image using an image restoration network to obtain a restored image comprises:

[0029] Based on the fuzzy degree model BDM, the fuzzy judgment of the query image is performed, and the formula is:

[0030] ,

[0031] in, and Represents the height and width of the image respectively, and Respectively Direction and The gradient of the image in the direction, express Direction and The sum of the number of gradient values ​​in the gradient map in the direction, represents the coordinates of the pixel, express Direction and The sum of the number of non-zero gradient values ​​in the gradient map in the direction;

[0032] If the BDM of a query image is greater than T, it is considered a clear image and no deblurring operation is required;

[0033] If its BDM<= T, it is considered to be a blurred image, and the image restoration network is used to restore the blurred image to obtain the restored image.

[0034] In a preferred embodiment, the steps of extracting features from the restored image based on the SuperPoint algorithm to obtain restored image feature information, and performing neighbor retrieval on the feature information using a retrieval algorithm based on an offline database include:

[0035] Based on the SuperPoint algorithm, feature extraction is performed on the restored image to obtain feature information of the restored image;

[0036] The Patch-NetVLAD image retrieval model is used to calculate the similarity score between the image pairs in the first matching area and the second matching area based on the pose labels in the offline database through the original NetVLAD descriptor to obtain spatial and appearance consistency, and obtain the first K similar images of the given restored image;

[0037] Obtain block descriptors of the first K similar images at different scales, perform local cross-matching of block descriptors between the first K similar images, obtain matching scores, and reorder the initial matching set according to the matching scores;

[0038] Based on the Patch-NetVLAD image retrieval model, starting from the most similar image retrieved, retrieval images are iteratively selected to obtain the final K groups of query-database image pairs.

[0039] In a preferred embodiment, the step of performing final positioning of the database, the query image, and all feature extraction and matching information using the epipolar geometry five-point method comprises:

[0040] Obtain the absolute pose and essential matrix information of the restored image in the database image;

[0041] Obtain d nearest neighbor image pairs of the restored image as well as feature information extracted from all images and feature matching information of the image pairs;

[0042] Traverse all image pairs in turn to obtain the essential matrix of the relative pose of each image pair , solved by the five-point method of epipolar geometry;

[0043] By decomposing the essential matrix through SVD, four relative postures are obtained, which are , , , ,in and By rotating around the baseline The conversion is performed, and the position of the restored image is estimated through triangulation based on the absolute pose of the database image and the relative pose of the image pair to complete the final positioning work.

[0044] The present invention also provides an indoor visual positioning system based on fuzzy images, comprising:

[0045] A first feature extraction module is used to set a plurality of reference points and collect a plurality of images as database images based on the positions of the reference points;

[0046] The database storage module is connected to the feature extraction module and is used to extract the features of the database image using the SuperPoint algorithm and the NetVLAD algorithm as database image feature information and save it;

[0047] The pose labels of the database images are obtained based on the reference points and stored corresponding to the database images to obtain an offline database;

[0048] An image deblurring module is used to determine a query image and judge whether the query image is a blurred image based on a blur degree model BDM. If the query image is a blurred image, an image restoration network is used to restore the blurred image to obtain a restored image.

[0049] The second feature extraction module is used to extract features of the restored image based on the SuperPoint algorithm to obtain feature information of the restored image, and to perform a nearest neighbor search on the feature information using a retrieval algorithm based on an offline database;

[0050] Image retrieval module, used to establish 2D-2D matching information between query image and database image by feature matching through SuperGlue algorithm, image pair and SuperPoint feature information;

[0051] The indoor positioning module is used to perform the final positioning work using the epipolar geometry five-point method on the database, query image and all feature extraction and matching information.

[0052] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0053] 1. The present invention extracts features from data images and query images based on the SuperPoint algorithm and the NetVLAD algorithm. NetVLAD features are used for image retrieval, and SuperPoint features are used for epipolar geometry five-point positioning. The advantages of the NetVLAD algorithm, which is fast and has high retrieval efficiency, and the SuperPoint algorithm, which has high accuracy and strong robustness, are fully utilized to improve the overall performance of the indoor positioning algorithm.

[0054] 2. The present invention uses SuperPoint and SuperGlue algorithms to achieve absolute pose estimation and relative pose estimation, obtains accurate positioning results, is flexible and lightweight, does not require additional input information and complex intermediate processes, and can be easily applied to new scenarios;

[0055] 3. The present invention can better divide the spatial position into multiple areas to obtain multiple matching areas, thereby improving the efficiency of matching image feature information to determine the image posture, greatly improving the efficiency and accuracy of image positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0057] Figure 1 It is a system block diagram of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] Example 1, please refer to Figure 1 As shown, the indoor visual positioning method based on fuzzy images described in this embodiment includes the following steps:

[0060] S1. The steps of setting the offline phase include:

[0061] S11, setting a plurality of reference points, and collecting a plurality of images as database images based on the positions of the reference points;

[0062] S12, using the SuperPoint algorithm and the NetVLAD algorithm to extract the features of the database image as database image feature information and save it;

[0063] S13, acquiring the pose label of the database image based on the reference point and storing the corresponding database image to obtain an offline database;

[0064] S2. The steps for setting up the online stage include:

[0065] S21, determining a query image, judging whether the query image is a blurred image based on a blur degree model BDM, and if the query image is a blurred image, restoring the blurred image using an image restoration network to obtain a restored image;

[0066] S22, extracting features of the restored image based on the SuperPoint algorithm to obtain feature information of the restored image, and performing a nearest neighbor search on the feature information using a retrieval algorithm based on an offline database;

[0067] S23, performing feature matching between the query image and the database image through the SuperGlue algorithm, the image pair and the SuperPoint feature information;

[0068] S24, an indoor positioning module that uses the epipolar geometry five-point method to perform final positioning on the database, the query image, and all feature extraction and matching information;

[0069] In one embodiment, the step S11 of setting a plurality of reference points and collecting a plurality of images as database images based on the reference point positions includes:

[0070] S111, obtaining indoor space information to be imaged, wherein the indoor space information includes indoor space structure and corresponding space dimensions;

[0071] S112, setting a plurality of reference points in the indoor space, and dividing the corresponding reference space according to the reference points to obtain a plurality of reference areas;

[0072] S113, collecting a plurality of images in the reference area based on the reference point positions as database images;

[0073] As described in the above steps S111-S113, the image to be processed here is an image captured indoors. In order to locate the indoor image and restore the blurred image, it is first necessary to set multiple reference points in the indoor space. In order to make the subsequent positioning and matching of the spatial image more accurate, it is necessary to set multiple reference points here, and each reference point has a corresponding reference area. The partitioning here is for more accurate subsequent search and matching of similar images, and better image processing and positioning;

[0074] In one embodiment, the step S12 of acquiring the pose label of the database image based on the reference point and storing the pose label corresponding to the database image to obtain the offline database includes:

[0075] S121, establishing a coordinate system in the reference area with the reference point as the origin, and obtaining the position and posture of the database image in the reference area according to the reference point;

[0076] S122, binding the posture with the corresponding image, and using the posture as a posture label of the image;

[0077] S123, storing the posture label, the corresponding image and the corresponding database image feature information as an offline database;

[0078] As described in the above steps S121-S123, a reference point is selected, and then the corresponding space is divided as a reference area according to the reference point. A coordinate system (X, Y coordinate system in the horizontal direction) is established in the area with the reference point as the origin. The posture corresponding to the acquired database image is recorded, and the posture is bound to the corresponding database image as the posture label of the image. Each image indicates the corresponding posture, and then the posture label, the corresponding image and the corresponding database image feature information are stored as an offline database. The offline database here is a positioning recognition that can only be performed according to the acquired database image. Its main purpose is to match an image in the online stage, which is more accurate when performing image positioning;

[0079] In one embodiment, the step S123 of storing the pose label, the corresponding image and the corresponding database image feature information as an offline database includes:

[0080] S1231, setting a corresponding number of storage bins corresponding to the number of reference areas, and setting an update bin corresponding to the storage bins;

[0081] S1232, obtaining the positional relationship between the reference areas, connecting the storage bins based on the positional relationship and establishing a reference area model of the storage bins for positional arrangement, and spatially associating the storage bins corresponding to the reference areas based on the line of sight channels and the reference points;

[0082] S1233, extracting image features from all reference areas, and binding information between the image features and the corresponding reference areas as a first matching area;

[0083] S1234, dividing the space in the storage bin with spatial association to obtain a plurality of associated spaces, and binding the image features with the associated spaces as a second matching area;

[0084] S1235, using the image features in the first matching area and the second matching area as matching tags, and the first matching area, the second matching area and the corresponding matching tags as database image feature information;

[0085] S1236, storing the pose label, the corresponding image, and the corresponding database image feature information as an offline database;

[0086] As described in the above steps S1231-S1236, a corresponding number of storage bins are set for the number of reference areas, and an update bin is set for the corresponding storage bin. The storage bin here is a storage bin set for storing all information in the reference area, and the update bin is used for adding image feature information and posture determined after the machine collects images in actual use to the corresponding storage bin, so as to realize the addition and update of images and image feature information in the storage bin, so that subsequent image collection can refer to more images, and can be continuously updated according to use. After that, the positional relationship between the reference areas is obtained, and the storage bins are connected according to the positional relationship of the reference areas, and a reference area model of the storage bins is established for position arrangement. The storage bins corresponding to the reference areas are spatially associated based on the line of sight channel and the reference point. The line of sight channel here is the space that the machine can capture in all directions at the reference point position. This may involve multiple reference areas or partial areas in multiple reference areas. For example, the middle position of the room is the reference point. In the absence of a room door, the reference point in the middle position of the room is If a full-scale photo is taken at the point, a partial image outside the door can also be taken. Then the space corresponding to the partial image outside the door can be spatially associated with the current reference point, and the image features in all reference areas are extracted (that is, the features extracted from the database image using the SuperPoint algorithm and the NetVLAD algorithm). The image features and the corresponding reference areas are bound as the first matching area, and the space in the storage bin with spatial association is divided to obtain multiple associated spaces. The image features and the associated spaces are bound as the second matching area, and the image features in the first matching area and the second matching area are used as matching labels. The first matching area, the second matching area and the corresponding matching labels are used as database image feature information. Finally, the posture label, the corresponding image and the corresponding database image feature information are stored as an offline database, which can better divide the spatial position into multiple areas and obtain multiple matching areas, thereby improving the efficiency of matching image feature information to determine the image posture, and greatly improving the efficiency and accuracy of image positioning;

[0087] In one embodiment, the step S21 of determining the query image, judging whether the query image is a blurred image based on the blur degree model BDM, and restoring the blurred image using an image restoration network to obtain a restored image if the query image is a blurred image, comprises:

[0088] Based on the fuzzy degree model BDM, the fuzzy judgment of the query image is performed, and the formula is:

[0089] ,

[0090] in, and Represents the height and width of the image respectively, and Respectively Direction and The gradient of the image in the direction, express Direction and The sum of the number of gradient values ​​in the gradient map in the direction, represents the coordinates of the pixel, express Direction and The sum of the number of non-zero gradient values ​​in the gradient map in the direction;

[0091] For all images, the smaller the BDM value, the more serious the blur. By setting the threshold T;

[0092] If the BDM of a query image is greater than T, it is considered a clear image and no deblurring operation is required;

[0093] If its BDM<= T, it is considered to be a blurred image, and the image restoration network is used to restore the blurred image to obtain the restored image.

[0094] In one embodiment, the step S22 of extracting features from the restored image based on the SuperPoint algorithm to obtain restored image feature information, and performing neighbor retrieval on the feature information using a retrieval algorithm based on an offline database includes:

[0095] S221, extracting features of the restored image based on the SuperPoint algorithm to obtain restored image feature information;

[0096] S222, using the Patch-NetVLAD image retrieval model to calculate the similarity score between the image pairs in the first matching area and the second matching area based on the pose labels in the offline database through the original NetVLAD descriptor to obtain spatial and appearance consistency, and obtain the first K similar images of the given restored image;

[0097] S223, obtaining block descriptors of the first K similar images at different scales, performing local cross matching of the block descriptors between the first K similar images to obtain matching scores, and reordering the initial matching set according to the matching scores;

[0098] S224, starting from the most similar image retrieved based on the Patch-NetVLAD image retrieval model, iteratively select retrieval images to obtain the final K groups of query-database image pairs;

[0099] As described in the above steps S221-S224, the Patch-NetVLAD model is used to perform image retrieval. The image feature information is used to match and search in the first matching area and the second matching area, so that the adjacent images can be matched quickly, and then the reference area can be determined to determine the image posture for image positioning. By calculating the similarity score between the image pairs, their spatial and appearance consistency is measured, and finally an image pair consisting of the query image and its first k neighboring images is obtained; the model adopts a hierarchical method, firstly, the original NetVLAD descriptor is used to retrieve the first k similar images of the given query image; then a new block descriptor is calculated for the top-ranked candidate images at multiple scales, and the block descriptors are locally cross-matched between the image pairs with geometric verification, so as to re-order the initial matching set according to the matching score and optimize the retrieval results; this scheme greatly reduces the computational cost brought by the cross-matching of block features while ensuring the retrieval accuracy, and adopts a visual positioning method based on the essential matrix. As long as the number of image pairs of the query image is not less than two pairs, its absolute posture hypothesis can be estimated from them.

[0100] In order to eliminate unreasonable estimates and make the algorithm robust to abnormal image pairs, the Random Sample Consensus (RANSAC) algorithm is applied in the positioning process. In each cycle, two image pairs are sampled. and , and based on them, a set of absolute poses of restored images are calculated Then we need to determine which image pairs are the internal pairs for this absolute pose hypothesis. Assuming that the database image is another neighboring image of the restored image, then according to the image pair The four relative positions between them can be obtained. After that, a relative rotation can be determined first , which makes the absolute rotation and Then, due to the absolute pose assumption Predicted from arrive The relative translation of , so you can use the angle The consistency between the predicted relative pose and the absolute pose hypothesis predicted by the image pair is measured. If the angle between the two predicted translation directions does not exceed a given threshold , then the image is As an internal pair, otherwise it is regarded as an external pair (abnormal pair). Finally, the RANSAC loop returns the absolute pose hypothesis with the largest number of internal pairs as the restored image The absolute pose prediction of

[0101] In one embodiment, the step S24 of performing final positioning work on the database, the query image, and all feature extraction and matching information using the epipolar geometry five-point method includes:

[0102] S241, obtaining the absolute position and essential matrix information of the restored image in the database image;

[0103] S242, obtaining d neighboring image pairs of the restored image and feature information extracted from all images and feature matching information of the image pairs;

[0104] S243, traverse all image pairs in turn to obtain the essential matrix of the relative position of each image pair , solved by the five-point method of epipolar geometry;

[0105] S244, decompose the essential matrix through SVD to obtain four relative postures, which are , , , ,in and By rotating around the baseline The conversion is performed, and the position of the restored image is estimated through triangulation based on the absolute pose of the database image and the relative pose of the image pair to complete the final positioning work;

[0106] As described in the above steps S241-S244, the database image features and the query image features are matched by the SuperGlue algorithm. These feature matching information will be used for the subsequent indoor positioning task. Then, positioning is performed by the positioning algorithm based on the essential matrix. Specifically, when locating the query image in a certain scene,

[0107] First, load the absolute pose and essential matrix information of the database image in this scene, the five nearest neighbor image pairs of all query images, and the feature point information extracted from all images and the feature matching information of the image pairs.

[0108] After that, we traverse all the image pairs in turn. For each image pair, we calculate the essential matrix encoding their relative pose , solved by the epipolar geometry five-point method.

[0109] Then, the essential matrix is ​​decomposed by SVD. Due to the ambiguity, four relative poses are obtained, which are , , , ,in and By rotating around the baseline Since the absolute pose of the database image is known, the position of the query image can be estimated by triangulation based on the absolute pose of the database image and the relative pose of the previously estimated image pair. To do this, it is sufficient to eliminate the ambiguity of the two rotations. When triangulating points, flip either direction The position of the point will not change, so as long as the number of image pairs of the query image satisfy , then the absolute position of the query image can be determined. and For two possible relative rotations (from the retrieved image The camera coordinate system is transformed to the query image camera coordinate system), and For the other two possible relative rotations, we get four absolute rotation predictions for the query image: , , , In actual calculation, because and ( and ) is about The two absolute pose predictions corresponding to the relative rotation with the smallest angle difference from each image pair are considered reasonable. Suppose a query image There are two retrieved neighbor images and , which constitutes two image pairs and , the absolute rotation of the query image is estimated based on these two image pairs ,and and is the corresponding relative rotation, and is the relative translation direction corresponding to it. Thus, two rays can be obtained, which are and ,in and are the scaling factors of the camera coordinate systems of the two images. Based on these two rays, the position of the query image can be determined by triangulation;

[0110] Example 2, please refer to Figure 1 As shown, an indoor visual positioning system based on fuzzy images includes:

[0111] A first feature extraction module is used to set a plurality of reference points and collect a plurality of images as database images based on the positions of the reference points;

[0112] The database storage module is connected to the feature extraction module and is used to extract the features of the database image using the SuperPoint algorithm and the NetVLAD algorithm as database image feature information and save it;

[0113] The pose labels of the database images are obtained based on the reference points and stored corresponding to the database images to obtain an offline database;

[0114] An image deblurring module is used to determine a query image and judge whether the query image is a blurred image based on a blur degree model BDM. If the query image is a blurred image, an image restoration network is used to restore the blurred image to obtain a restored image.

[0115] The second feature extraction module is used to extract features of the restored image based on the SuperPoint algorithm to obtain feature information of the restored image, and to perform a nearest neighbor search on the feature information using a retrieval algorithm based on an offline database;

[0116] Image retrieval module, used to establish 2D-2D matching information between query image and database image by feature matching through SuperGlue algorithm, image pair and SuperPoint feature information;

[0117] The indoor positioning module is used to perform the final positioning work using the epipolar geometry five-point method on the database, query image and all feature extraction and matching information.

[0118] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for indoor visual positioning based on fuzzy images, characterized in that: The following steps are involved: The steps to set up the offline phase include: Setting a plurality of reference points, and acquiring a plurality of images as database images based on the positions of the reference points; Use SuperPoint algorithm and NetVLAD algorithm to extract the features of database images as database image feature information and save them; The pose labels of the database images are obtained based on the reference points and stored corresponding to the database images to obtain an offline database; The step of acquiring the pose label of the database image based on the reference point and storing the corresponding database image to obtain the offline database includes: A coordinate system is established in the reference area using the reference point as the origin, and the position and pose of the database image in the reference area is obtained according to the reference point; Bind the pose to the corresponding image, and use the pose as the pose label of the image; The pose labels, the corresponding images and the corresponding database image feature information are stored as an offline database; The step of storing the pose label, the corresponding image and the corresponding database image feature information as an offline database includes: A corresponding number of storage bins are set corresponding to the number of reference areas, and an update bin is set corresponding to the storage bins; Obtain the positional relationship between the reference areas, connect the storage bins based on the positional relationship and establish a reference area model of the storage bins for position arrangement, and spatially associate the storage bins corresponding to the reference areas based on the line of sight channels and reference points; Extracting image features from all reference areas, and binding information between the image features and the corresponding reference areas as the first matching area; Dividing the space in the storage bin with spatial association to obtain a plurality of association spaces, and binding the image features with the association spaces as the second matching area; The image features in the first matching area and the second matching area are used as matching labels, and the first matching area, the second matching area and the corresponding matching labels are used as database image feature information; The pose labels, the corresponding images and the corresponding database image feature information are stored as an offline database; The steps to set up the online stage include: Determine a query image, and judge whether the query image is a blurred image based on a blur degree model BDM. If the query image is a blurred image, use an image restoration network to restore the blurred image to obtain a restored image. The restored image feature information is obtained by extracting features based on the SuperPoint algorithm, and the feature information is retrieved by using a retrieval algorithm based on the offline database to perform a nearest neighbor retrieval. The 2D-2D matching information between the query image and the database image is established by feature matching using the SuperGlue algorithm, image pairs and SuperPoint feature information; The database, query image and all feature extraction and matching information are finally positioned using the epipolar geometry five-point method.

2. The indoor visual positioning method based on fuzzy images according to claim 1, characterized in that: The step of setting a plurality of reference points and collecting a plurality of images as database images based on the positions of the reference points comprises: Acquire indoor space information to be captured by an image, wherein the indoor space information includes an indoor space structure and a corresponding space size; A plurality of reference points are set in the indoor space, and corresponding reference spaces are divided according to the reference points to obtain a plurality of reference areas; A plurality of images are acquired in the reference area based on the reference point positions as database images.

3. The indoor visual positioning method based on fuzzy images according to claim 1, characterized in that: The step of determining a query image, judging whether the query image is a blurred image based on a blur degree model BDM, and if the query image is a blurred image, restoring the blurred image using an image restoration network to obtain a restored image comprises: Based on the fuzzy degree model BDM, the fuzzy judgment of the query image is performed, and the formula is: Where H and W represent the height and width of the image respectively, g x (i,j)=|f(i+1,j)-f(i,j)| and g y (i,j)=|f(i,j+1)-f(i,j)| represents the gradient of the image in the X direction and the Y direction respectively, N represents the sum of the number of gradient values ​​in the gradient map in the X direction and the Y direction, (i,j) represents the coordinates of the pixel, G num Represents the sum of the number of non-zero gradient values ​​in the gradient map in the X direction and the Y direction; If the BDM of a query image is greater than T, it is considered a clear image and no deblurring operation is required; If its BDM<=T, it is considered to be a blurred image, and the blurred image is restored using an image restoration network to obtain a restored image.

4. The indoor visual positioning method based on fuzzy images according to claim 1, characterized in that: The steps of extracting features from the restored image based on the SuperPoint algorithm to obtain restored image feature information, and performing neighbor retrieval on the feature information using a retrieval algorithm based on an offline database include: Based on the SuperPoint algorithm, feature extraction is performed on the restored image to obtain feature information of the restored image; The Patch-NetVLAD image retrieval model is used to calculate the similarity score between the image pairs in the first matching area and the second matching area based on the pose labels in the offline database through the original NetVLAD descriptor to obtain spatial and appearance consistency, and obtain the first K similar images of the given restored image; Obtain block descriptors of the first K similar images at different scales, perform local cross-matching of block descriptors between the first K similar images, obtain matching scores, and reorder the initial matching set according to the matching scores; Based on the Patch-NetVLAD image retrieval model, starting from the most similar image retrieved, retrieval images are iteratively selected to obtain the final K groups of query-database image pairs.

5. The indoor visual positioning method based on fuzzy images according to claim 1, characterized in that: The step of using the epipolar geometry five-point method to perform final positioning of the database, the query image, and all feature extraction and matching information includes: Obtain the absolute pose and essential matrix information of the restored image in the database image; Obtain d nearest neighbor image pairs of the restored image as well as feature information extracted from all images and feature matching information of the image pairs; Traverse all image pairs in turn to obtain the essential matrix E of the relative pose of each image pair, and solve it using the epipolar geometry five-point method; By decomposing the essential matrix through SVD, four relative poses are obtained, namely (R, t), (R, -t), (R ′ ,t),(R ′ ,-t), where R and R ′ The transformation can be performed by rotating 180° around the baseline, and the final positioning is completed by estimating the position of the restored image through triangulation based on the absolute pose of the database image and the relative pose of the image pair.

6. An indoor visual positioning system based on fuzzy images, used to implement an indoor visual positioning method based on fuzzy images as described in any one of claims 1 to 5, characterized in that: include: A first feature extraction module is used to set a plurality of reference points and collect a plurality of images as database images based on the positions of the reference points; The database storage module is connected to the feature extraction module and is used to extract the features of the database image using the SuperPoint algorithm and the NetVLAD algorithm as database image feature information and save it; A corresponding storage module is linked to the database storage module, and is used to obtain the pose label of the database image based on the reference point and store the corresponding database image to obtain an offline database; An image deblurring module is used to determine a query image and judge whether the query image is a blurred image based on a blur degree model BDM. If the query image is a blurred image, an image restoration network is used to restore the blurred image to obtain a restored image. The second feature extraction module is used to extract features of the restored image based on the SuperPoint algorithm to obtain feature information of the restored image, and to perform a nearest neighbor search on the feature information using a retrieval algorithm based on an offline database; Image retrieval module, used to establish 2D-2D matching information between query image and database image by feature matching through SuperGlue algorithm, image pair and SuperPoint feature information; The indoor positioning module is used to perform the final positioning work using the epipolar geometry five-point method on the database, query image and all feature extraction and matching information.

Images