A method for fast retrieval of indoor images with illumination robustness

Abstract

The application discloses an indoor image fast retrieval method with illumination robustness, and comprises the following steps: inputting a standard query image into a trained feature embedding extraction model to extract features and obtain a query embedding vector; respectively calculating the similarity of the query embedding vector and each vector in a reference embedding vector set obtained in advance to obtain a similarity score corresponding to each reference image; arranging the similarity scores of the reference images in descending order, and screening the reference images corresponding to the top K similarity scores as a candidate matching list; performing confidence evaluation on the candidate matching list, and outputting the final retrieval result of the reference image corresponding to the first ranking in the candidate matching list or the final retrieval result obtained by weighted fusion of the retrieval results of the top K reference images in the candidate matching list. The application can achieve high robustness and high efficient retrieval under significant illumination changes.

Description

Technical Field

[0001] This invention relates to the field of image retrieval technology, and in particular to a fast indoor image retrieval method with illumination robustness. Background Technology

[0002] With the widespread application of indoor inspections and autonomous mobile devices, the problem of GPS signals being easily blocked and interfered with has become increasingly prominent. As a core support for positioning and scene recognition, the stability and adaptability of image retrieval technology are particularly important. Currently, image retrieval in indoor scenes faces many pain points. Traditional retrieval methods either rely on traversing and matching one by one, which is inefficient and cannot meet the needs of large-scale scenarios; or they lack the ability to adapt to changes in lighting, and are prone to matching deviations under complex lighting conditions such as strong light and shadow, resulting in insufficient retrieval accuracy.

[0003] Currently, the known technologies for indoor image retrieval are mainly divided into two mainstream implementation schemes: global descriptor extraction and FFT phase correlation. The former relies on deep network architectures such as convolutional neural networks and Transformers to complete global feature encoding, and then completes matching through similarity calculation or retrieval index. The latter uses image cross power spectrum to achieve image retrieval in the frequency domain. Although both technologies have mature architectures and are widely used, neither can simultaneously achieve high retrieval efficiency, strong illumination robustness, and millisecond-level real-time response performance in large-scale scenarios. In addition, existing technologies have explored optimization from different directions, such as geometric feature clustering and fusion, traditional image processing illumination correction, and multi-transformation multi-database fusion, but none of them have been able to effectively overcome the core bottlenecks: some solutions rely too much on fixed geometric structure features and do not introduce an adaptive suppression mechanism for illumination interference, resulting in a high matching failure rate and a lengthy multi-stage calculation process under dynamic illumination; some solutions rely on manually designed traditional feature operators, which have low feature discrimination, are computationally intensive and have weak generalization, and cannot cope with extreme illumination changes or be deeply integrated with efficient vector databases; some solutions improve illumination adaptability through multi-branch parallel processing and multi-database fusion, but this significantly increases storage and computational overhead, and does not fundamentally construct illumination-invariant feature embeddings, so the recall and accuracy are still significantly insufficient under significant illumination changes.

[0004] In summary, existing indoor image retrieval technologies generally fail to effectively balance computational efficiency, illumination robustness, and real-time performance, thus failing to meet the practical application needs of large-scale, complex indoor scenes. Therefore, there is an urgent need to develop a fast indoor image retrieval method with illumination robustness, systematically optimizing the entire chain from feature extraction, model training, to retrieval matching. This would achieve millisecond-level image retrieval with strong illumination invariance, low computational overhead, and large-scale applicability, providing stable and reliable visual positioning and scene recognition support for indoor autonomous mobile devices. Summary of the Invention

[0005] The purpose of this invention is to provide a fast indoor image retrieval method with illumination robustness, which achieves high robustness and efficient retrieval under conditions of significant illumination changes.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] In a first aspect, the present invention provides a fast indoor image retrieval method with illumination robustness, comprising:

[0008] Get the image to be queried;

[0009] The image to be queried is preprocessed to obtain the standard query image;

[0010] The standard query image is input into the trained feature embedding extraction model for feature extraction to obtain the query embedding vector;

[0011] The similarity between the query embedding vector and each vector in the pre-acquired set of reference embedding vectors is calculated to obtain the similarity score for each reference image; the set of reference embedding vectors corresponds one-to-one with the reference images in the reference image database;

[0012] The reference images are sorted in descending order of their similarity scores, and the reference images corresponding to the top K similarity scores are selected as the candidate matching list.

[0013] The confidence level of the candidate matching list is evaluated. If the confidence level evaluation result meets the preset conditions, the final retrieval result corresponding to the first ranked reference image in the candidate matching list is directly output.

[0014] If the confidence assessment result does not meet the preset conditions, the retrieval results of the first K reference images in the candidate matching list are weighted and fused, and the final fusion result is output.

[0015] Optionally, the method for constructing the reference embedding vector set includes:

[0016] Construct a reference image database; the reference image database includes several reference images;

[0017] Several reference images in the reference image database are preprocessed to obtain several standard reference images;

[0018] The trained feature embedding extraction model is used to extract features from several standard reference images sequentially to obtain corresponding reference embedding vectors.

[0019] A set of reference embedding vectors is constructed based on the corresponding reference embedding vectors of several standard reference images.

[0020] Optionally, the preprocessing includes:

[0021] The original image is resized to a first preset size to obtain the adjusted image; the original image is the image to be queried or a reference image.

[0022] The adjusted central region of the image is cropped to a second preset size to obtain the cropped image; the second preset size is consistent with the input size of the feature embedding extraction model;

[0023] The cropped image is standardized to obtain the preprocessed original image.

[0024] Optionally, the feature embedding extraction model includes: a backbone network, a CBAM attention module, and a post-processing module connected in sequence;

[0025] The backbone network is used to extract multi-scale features from the input image to obtain the highest-level semantic feature map;

[0026] The CBAM attention module is used to recalibrate the features of the highest-level semantic feature map, strengthen key features and suppress irrelevant features to obtain a recalibrated feature map.

[0027] The post-processing module is used to sequentially perform global pooling, feature projection, and normalization on the recalibrated feature map to obtain the embedding vector corresponding to the input image.

[0028] Optionally, the backbone network may be any one of EfficientNet-B0, ResNet-18, ShuffleNetV2, and MobileNetV3-Small.

[0029] Optionally, the CBAM attention module includes a channel attention submodule and a spatial attention submodule connected in sequence;

[0030] The channel attention submodule can be replaced by the SE channel attention module, the ECA channel attention module, or a module that only uses global average pooling to generate channel weights;

[0031] The spatial attention submodule can be replaced by a deformable convolution module;

[0032] The connection order of the channel attention submodule and the spatial attention submodule can be adjusted.

[0033] Optionally, the post-processing module includes a global pooling module, a feature projection module, a normalization module, and an L2 normalization module connected in sequence.

[0034] The global pooling module can be any one of the following: global average pooling module, global max pooling module, or a weighted fusion module of global average pooling and global max pooling.

[0035] The feature projection module employs a single-layer linear layer or a multi-layer perceptron.

[0036] The normalization module can be any one of layer normalization module, batch normalization module, or group normalization module.

[0037] Optionally, the similarity calculation method is selected from any one of cosine similarity, Euclidean distance, or weighted inner product.

[0038] Optionally, the training process of the feature embedding extraction model includes:

[0039] Obtain a training sample set, wherein the format of the sample images in the training sample set is the same as the format of the standard query image;

[0040] Data augmentation processing is performed on sample images in the training sample set to generate multiple different augmented views of the same sample image; the data augmentation processing includes color jitter, random rotation, and affine transformation;

[0041] Multiple augmented views of the same sample image are used as positive sample pairs, and multiple augmented views of different images are used as negative sample pairs. These are input into the feature embedding extraction model to be trained to extract the embedding vectors corresponding to each augmented view.

[0042] The loss value is calculated using a contrastive learning loss function, which may be either InfoNCE loss or Triplet loss.

[0043] The model parameters are optimized by backpropagation based on the loss value until the model converges, resulting in a well-trained feature embedding extraction model.

[0044] Optionally, before performing similarity calculations between the query embedding vector and each vector in the pre-obtained set of reference embedding vectors, a retrieval index is pre-constructed for the set of reference embedding vectors; the retrieval index includes an exact retrieval index or an approximate nearest neighbor retrieval index, the exact retrieval index includes a Flat index, and the approximate nearest neighbor retrieval index includes an HNSW index and an IVF index.

[0045] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0046] This invention provides a fast indoor image retrieval method with illumination robustness. This method effectively solves the core problem of balancing computational efficiency, illumination robustness, and real-time performance in existing technologies through a dual-strategy output approach involving preprocessing, feature embedding extraction, similarity ranking, and confidence-driven output. Under high confidence, the Top-1 result is directly output, i.e., the reference image ranked first in the candidate matching list, ensuring real-time retrieval. Under low confidence, Top-K weighted fusion is triggered, i.e., the retrieval results of the top K reference images in the candidate matching list are weighted and fused to improve retrieval accuracy. This method can effectively correct matching errors caused by drastic changes in illumination in a single frame image, significantly improving retrieval accuracy and recall in complex lighting scenarios.

[0047] This invention provides a fast indoor image retrieval method with illumination robustness. The method uses a feature embedding extraction model for feature embedding extraction. The backbone network of the feature embedding extraction module is followed by a CBAM attention module, which can adaptively suppress the feature weights of areas with illumination interference, greatly improving the illumination robustness of feature extraction and effectively reducing the matching failure rate under dynamic illumination conditions such as strong backlight and shadow occlusion. At the same time, it supports flexible combinations of various attention sub-modules and connection orders, taking into account the accuracy and speed requirements of different scenarios.

[0048] This invention provides a fast indoor image retrieval method with robustness to illumination. In this method, the training of the feature embedding extraction model is carried out through a targeted data augmentation and contrastive learning training paradigm that includes color jitter, random rotation, and affine transformation. Different augmented views of the same image are used as positive sample pairs, enabling the model to learn the essential semantic features of the scene that are independent of illumination and viewpoint from the root, which greatly improves the robustness of the embedding vector to illumination changes and viewpoint shifts. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 The diagram shown is a schematic of a fast indoor image retrieval process with illumination robustness according to the present invention.

[0051] Figure 2 The diagram shown is a flowchart of a fast indoor image retrieval process with illumination robustness in one embodiment of the present invention.

[0052] Figure 3 The diagram shown is a schematic diagram of the feature embedding extraction model structure in one embodiment of the present invention;

[0053] Figure 4 The diagram shown is a schematic representation of the confidence assessment process in one embodiment of the present invention. Detailed Implementation

[0054] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0055] Example 1

[0056] like Figure 1 and Figure 2 As shown in the figure, this invention provides a fast indoor image retrieval method with illumination robustness, including the following steps:

[0057] S1: Obtain the image to be queried;

[0058] S2: Preprocess the image to be queried to obtain the standard query image;

[0059] S3: Input the standard query image into the trained feature embedding extraction model to extract features and obtain the query embedding vector;

[0060] S4: Calculate the similarity between the query embedding vector and each vector in the pre-obtained set of reference embedding vectors to obtain the similarity score for each reference image; the set of reference embedding vectors corresponds one-to-one with the reference images in the reference image database;

[0061] S5: Sort the reference images in descending order according to their similarity scores, and select the reference images corresponding to the top K similarity scores as the candidate matching list;

[0062] S6: Evaluate the confidence level of the candidate matching list. If the confidence level evaluation result meets the preset conditions, directly output the final retrieval result corresponding to the first ranked reference image in the candidate matching list.

[0063] If the confidence assessment result does not meet the preset conditions, the retrieval results of the first K reference images in the candidate matching list are weighted and fused, and the final fusion result is output.

[0064] The methods for constructing the reference embedding vector set include:

[0065] S01: Construct a reference image database; the reference image database includes several reference images;

[0066] S02: Preprocess several reference images from the reference image database to obtain several standard reference images;

[0067] S03: Using the trained feature embedding extraction model, feature extraction is performed sequentially on several standard reference images to obtain corresponding reference embedding vectors;

[0068] S04: Construct a set of reference embedding vectors based on the corresponding reference embedding vectors of several standard reference images.

[0069] In this embodiment, the preprocessing process in steps S1 and S02 includes:

[0070] The original image is resized to a first preset size to obtain the adjusted image; the original image is the image to be queried or a reference image.

[0071] The adjusted central region of the image is cropped to a second preset size to obtain the cropped image; the second preset size is consistent with the input size of the feature embedding extraction model;

[0072] The cropped image is standardized to obtain the preprocessed original image.

[0073] Specifically, the first preset size is 256×256, and the second preset size is 224×224. The image to be queried is first resized to 256×256, then cropped to 224×224, and standardized using the mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225].

[0074] In this embodiment, the feature embedding extraction model in step S3 includes: a backbone network, a CBAM attention module, and a post-processing module connected in sequence;

[0075] The backbone network is used to extract multi-scale features from the input image to obtain the highest-level semantic feature map. Furthermore, the backbone network can be any one of EfficientNet-B0, ResNet-18, ShuffleNetV2, and MobileNetV3-Small. These networks differ significantly in structure, computational cost, and feature representation capabilities, allowing for targeted adaptation to different deployment computing power and complex indoor lighting scenarios. Among them, MobileNetV3-Small and ShuffleNetV2 are lightweight networks. Relying on lightweight designs such as depthwise separable convolution and channel shuffling, they can significantly compress the number of parameters and computation, resulting in high computational efficiency and low inference latency. They can effectively solve the problems of poor real-time performance and high computational overhead in existing technologies, and are suitable for deployment on low-computing-power devices such as inspection robots and embedded terminals. ResNet-18 relies on the residual connection structure, which has stronger feature representation capabilities and better generalization. It can stably extract basic features that resist interference, weaken feature perturbations caused by indoor shadows and local brightness differences, and balance retrieval accuracy with moderate computational cost. EfficientNet-B0 has stronger feature mining capabilities in complex scenes through collaborative scaling optimization of width, depth and resolution. It has higher semantic discriminative power and can effectively cope with extreme conditions such as strong backlight and uneven lighting indoors, making up for the shortcomings of traditional lightweight networks in feature extraction under complex lighting conditions. This invention allows for flexible replacement of equipment based on the scale of the indoor scene, the computing power of the devices, and the complexity of the lighting. It achieves a flexible balance between lighting robustness, computing efficiency, and real-time retrieval, adapting to the needs of large-scale indoor image library retrieval and diverse edge deployments.

[0076] The CBAM attention module is used to recalibrate the features of the highest-level semantic feature map, strengthening key features and suppressing irrelevant features to obtain a recalibrated feature map. It includes a channel attention submodule and a spatial attention submodule connected in sequence. The channel attention submodule can be replaced by an SE channel attention module, an ECA channel attention module, or a module that only uses global average pooling to generate channel weights. The spatial attention submodule can be replaced by a deformable convolutional module. The connection order of the channel attention submodule and the spatial attention submodule can be adjusted, i.e., spatial attention first, then channel attention. The CBAM attention module, SE channel attention module, and ECA channel attention module are all technical features known to those skilled in the art and will not be described in detail here.

[0077] The post-processing module is used to sequentially perform global pooling, feature projection, and normalization on the recalibrated feature map to obtain the embedding vector corresponding to the input image. Further, the post-processing module includes a global pooling module, a feature projection module, a normalization module, and an L2 normalization module connected in sequence; the global pooling module uses any one of a global average pooling module, a global max pooling module, or a weighted fusion module of global average pooling and global max pooling; the feature projection module uses a single-layer linear layer or a multi-layer perceptron; the normalization module uses any one of a layer normalization module, a batch normalization module, or a group normalization module. In this invention, the post-processing module, by sequentially executing a standardized post-processing flow of global pooling, feature projection, and normalization, can convert the recalibrated feature map into a compact embedding vector of fixed dimensions, eliminating the overall brightness difference between different images, and further improving the illumination invariance and discriminative power of the feature vector; at the same time, the compact vector representation significantly reduces the overhead of subsequent similarity calculation and storage, improving overall computational efficiency.

[0078] Specifically, the feature embedding extraction model takes MobileNetV3-Smal+CBAM+global average pooling module+projection module+layer normalization module+L2 normalization module as an example, such as... Figure 3 As shown;

[0079] CBAM is inserted after the highest-level semantic feature map, which is a 576×7×7 feature map output by MobileNetV3-Small. Channel attention is generated by two fully connected layers (GAP / GMP) to produce weights; spatial attention is generated by a 7×7 convolution to produce a weight map; these are applied in the order of channel attention first, then spatial attention.

[0080] Specifically, global average pooling (GAP) and global max pooling (GMP) are performed on the high-level semantic feature maps to obtain two 576-dimensional vectors. These vectors are then processed through a shared two-layer fully connected network (MLP) to generate channel weights Wc. The channel weight coefficients are then multiplied element-wise with the high-level semantic feature maps to perform feature recalibration along the channel dimensions, adaptively enhancing effective semantic channels and suppressing noisy channels. The spatial attention submodule, after channel weighting, performs global average pooling and global max pooling simultaneously along the channel dimensions of the feature maps, concatenating them to obtain a 2×7×7 tensor. This tensor is then convolved with a 7×7 layer to generate spatial weights Ws, which are then multiplied back into the output feature map of the channel attention module. Channel attention is applied before spatial attention; channel importance is optimized globally first, followed by fine-tuning of the spatial distribution.

[0081] The processing flow is as follows:

[0082] The input image is fed into the trained MobileNetV3-Small backbone network, which is processed by depthwise separable convolution, inverted residual blocks, and h-swish activation function to output a 576×7×7 feature map.

[0083] After the 576×7×7 feature map is enhanced by the CBAM attention module, it is compressed into a 576-dimensional vector by the global average pooling module, mapped to a 256-dimensional embedding space by the projection module, and then normalized by the layer normalization module (LayerNorm stable distribution) and L2 normalized to output a unit vector, which is the embedding vector corresponding to the input image, ensuring that the similarity depends only on the direction.

[0084] In this embodiment, in step S4, the similarity between the query embedding vector and each vector in the pre-acquired set of reference embedding vectors is calculated. The similarity calculation method is selected from any one of cosine similarity, Euclidean distance or weighted inner product.

[0085] This embodiment uses cosine similarity as the core metric, and its mathematical expression is:

[0086] ;

[0087] in, Represents the query embedding vector; This represents the i-th reference embedding vector;

[0088] Given that all embedding vectors have undergone L2 normalization during the extraction phase, i.e. Therefore, the above formula can be simplified to an inner product form:

[0089] .

[0090] Specifically, before calculating the similarity between the query embedding vector and each vector in the pre-acquired set of reference embedding vectors, a retrieval index is pre-constructed for the set of reference embedding vectors. The retrieval index includes an exact retrieval index or an approximate nearest neighbor retrieval index. The exact retrieval index includes the Flat index, i.e., for a medium-sized reference library, the FaissIndexFlatIP can be used to achieve an exact inner product search. The approximate nearest neighbor retrieval index includes the HNSW index and the IVF index, i.e., for a large-scale library, the IndexIVFFlat or IndexHNSW and other approximate nearest neighbor indexes are used to significantly reduce computational overhead while maintaining a high recall rate. In very small-scale scenarios, the complete similarity matrix can be calculated directly through matrix multiplication.

[0091] After similarity calculation, the scores are sorted in descending order to obtain the sorted reference image index sequence. A top-K matching list, i.e. a candidate matching list, is generated based on this sequence. The K value can be dynamically adjusted according to the application scenario. The original index, similarity score, and optional pre-associated metadata of each candidate are retained to support extended applications.

[0092] In this embodiment, step S6: perform confidence evaluation on the candidate matching list. If the confidence evaluation result meets the preset conditions, the final retrieval result corresponding to the first ranked reference image in the candidate matching list is directly output.

[0093] If the confidence assessment result does not meet the preset conditions, the retrieval results of the first K reference images in the candidate matching list are weighted and fused, and the final fusion result is output.

[0094] The candidate matching list includes the reference image index, similarity score, and optional pre-associated metadata (such as image path, scene label, etc.) for each candidate, which is prepared and labeled in advance during the reference image library construction phase.

[0095] Specifically, such as Figure 4 As shown, the processing flow is as follows:

[0096] First, the best match is selected, using the top-1 candidate with the highest similarity by default. Its index, similarity score, and associated metadata are directly output as the recognition result. This method has extremely low latency in high-confidence scenarios and is suitable for applications with extremely high real-time requirements.

[0097] To balance accuracy and reliability, this invention integrates a confidence assessment mechanism: It calculates the top-1 similarity score; if the score is not less than a preset threshold θ, the top-1 output is used directly; if it is less than the threshold θ, it switches to a top-K weighted fusion strategy. Fusion is based on the similarity score, with weights calculated using a softmax function for normalization, and finally, weighted fusion is performed to obtain the fusion result, further improving the robustness of recognition under uncertain conditions such as illumination changes, noise, or viewpoint shifts.

[0098] In this embodiment, an IMU inertial data smoothing step can be inserted between steps S3 and S4. That is, for the query embedding vector output by the feature embedding extraction model, the inter-frame pose change information output by the IMU is first used to perform temporal weighted fusion with the historical frame embedding vector through Kalman filtering or first-order exponential low-pass filtering to obtain a smoothed query embedding vector. Then, the subsequent similarity calculation is performed to eliminate the single-frame feature drift caused by drastic changes in illumination in a short period of time from the feature level.

[0099] In this embodiment, a LiDAR point cloud-assisted verification step is inserted before the confidence assessment begins. Specifically, for the top K candidate reference images obtained from similarity ranking, the real-time LiDAR point cloud features of the current frame are extracted and their similarity is calculated with the pre-stored LiDAR feature labels associated with each candidate reference image. The visual similarity and LiDAR similarity are fused according to a preset weight to generate a comprehensive similarity and the candidate list is re-ranked before proceeding to the subsequent confidence assessment and dual-strategy output stage. This filters out purely visual mismatches from the result level. Without significantly increasing inference latency, the retrieval robustness under extreme lighting and texture loss scenarios is greatly improved.

[0100] In this embodiment, the training process of the feature embedding extraction model includes:

[0101] Obtain a training sample set, wherein the format of the sample images in the training sample set is the same as the format of the standard query image;

[0102] Data augmentation processing is performed on sample images in the training sample set to generate multiple different augmented views of the same sample image; the data augmentation processing includes color jitter, random rotation, and affine transformation;

[0103] Multiple augmented views of the same sample image are used as positive sample pairs, and multiple augmented views of different images are used as negative sample pairs. These are input into the feature embedding extraction model to be trained to extract the embedding vectors corresponding to each augmented view.

[0104] The loss value is calculated using a contrastive learning loss function, which may be either InfoNCE loss or Triplet loss.

[0105] The model parameters are optimized by backpropagation based on the loss value until the model converges, resulting in a well-trained feature embedding extraction model.

[0106] Specifically, this embodiment uses a targeted data augmentation and contrastive learning training paradigm that includes color jitter, random rotation, and affine transformation to enable the model to learn the essential semantic features of the scene from the root, which are independent of lighting and viewpoint, thus greatly improving the robustness of the embedded vector to lighting changes and viewpoint shifts.

[0107] Example 2

[0108] This invention provides an indoor image retrieval system with illumination robustness, comprising:

[0109] The data acquisition module is used to acquire the image to be queried;

[0110] The preprocessing module is used to preprocess the image to be queried to obtain a standard query image;

[0111] The feature extraction module is used to input the standard query image into the trained feature embedding extraction model for feature extraction to obtain the query embedding vector;

[0112] The similarity calculation module is used to calculate the similarity between the query embedding vector and each vector in the pre-acquired set of reference embedding vectors to obtain the similarity score corresponding to each reference image; the set of reference embedding vectors corresponds one-to-one with the reference images in the reference image database;

[0113] The sorting module is used to sort the reference images in descending order according to their similarity scores and select the reference images corresponding to the top K similarity scores as the candidate matching list.

[0114] The evaluation module is used to evaluate the confidence level of the candidate matching list. If the confidence level evaluation result meets the preset conditions, the final retrieval result corresponding to the first ranked reference image in the candidate matching list is directly output. If the confidence level evaluation result does not meet the preset conditions, the retrieval results of the first K reference images in the candidate matching list are weighted and fused, and the fused final retrieval result is output.

[0115] The specific functions of each module described above are explained in the relevant content of the method in Embodiment 1, and will not be repeated here.

[0116] Example 3

[0117] Based on the illumination-robust indoor image fast retrieval method and system mentioned in Examples 1 and 2, this example proposes the following experimental verification:

[0118] Validation was performed using the same hardware / dataset as the baseline (MobileNetV3 / FFT). The dataset was a self-built indoor image library containing 8-bit images with significant grayscale differences and substantial illumination variations, which could be used to test the algorithm's illumination robustness. The images in the database were categorized into training (10492 images), validation (2244 images), and test (2258 images). The Gallery (2258 images) and Query (50 images) were independent of the training and validation sets to ensure no data leakage during the evaluation process and to maintain the fairness and objectivity of the experiment.

[0119] Table 1: Performance Comparison Experiment Results of Different Methods

[0120]

[0121] The results are shown in the table above. As can be seen from the table, the present invention is superior in both robustness and efficiency, verifying the effectiveness of the core mechanism.

[0122] Example 4

[0123] This embodiment describes an electronic device, including: a memory and a processor;

[0124] The memory is used to store programs;

[0125] The processor is used to execute the program to implement the light-robust indoor image fast retrieval method in Embodiment 1.

[0126] Example 5

[0127] This embodiment also introduces a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the light-robust indoor image fast retrieval method in Embodiment 1.

[0128] Example 6

[0129] This embodiment also introduces a computer program product, including a computer program that, when executed by a processor, implements the light-robust indoor image fast retrieval method in Embodiment 1.

[0130] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A fast indoor image retrieval method with illumination robustness, characterized in that, include: Get the image to be queried; The image to be queried is preprocessed to obtain the standard query image; The standard query image is input into the trained feature embedding extraction model for feature extraction to obtain the query embedding vector; The similarity between the query embedding vector and each vector in the pre-acquired set of reference embedding vectors is calculated to obtain the similarity score for each reference image; the set of reference embedding vectors corresponds one-to-one with the reference images in the reference image database; The reference images are sorted in descending order of their similarity scores, and the reference images corresponding to the top K similarity scores are selected as the candidate matching list. The confidence level of the candidate matching list is evaluated. If the confidence level evaluation result meets the preset conditions, the final retrieval result corresponding to the first ranked reference image in the candidate matching list is directly output. If the confidence assessment result does not meet the preset conditions, the retrieval results of the first K reference images in the candidate matching list are weighted and fused, and the final fusion result is output.

2. The indoor image retrieval method with illumination robustness according to claim 1, characterized in that, The method for constructing the reference embedding vector set includes: Construct a reference image database; the reference image database includes several reference images; Several reference images in the reference image database are preprocessed to obtain several standard reference images; The trained feature embedding extraction model is used to extract features from several standard reference images sequentially to obtain corresponding reference embedding vectors. A set of reference embedding vectors is constructed based on the corresponding reference embedding vectors of several standard reference images.

3. The indoor image retrieval method with illumination robustness according to claim 2, characterized in that, The preprocessing includes: The original image is resized to a first preset size to obtain the adjusted image; the original image is the image to be queried or a reference image. The adjusted central region of the image is cropped to a second preset size to obtain the cropped image; the second preset size is consistent with the input size of the feature embedding extraction model; The cropped image is standardized to obtain the preprocessed original image.

4. The indoor image retrieval method with illumination robustness according to claim 3, characterized in that, The feature embedding extraction model includes: a sequentially connected backbone network, a CBAM attention module, and a post-processing module; The backbone network is used to extract multi-scale features from the input image to obtain the highest-level semantic feature map; The CBAM attention module is used to recalibrate the features of the highest-level semantic feature map, strengthen key features and suppress irrelevant features to obtain a recalibrated feature map. The post-processing module is used to sequentially perform global pooling, feature projection, and normalization on the recalibrated feature map to obtain the embedding vector corresponding to the input image.

5. The indoor image retrieval method with illumination robustness according to claim 4, characterized in that, The backbone network adopts any one of EfficientNet-B0, ResNet-18, ShuffleNetV2, and MobileNetV3-Small.

6. The indoor image retrieval method with illumination robustness according to claim 5, characterized in that, The CBAM attention module includes a channel attention submodule and a spatial attention submodule connected in sequence; The channel attention submodule can be replaced by the SE channel attention module, the ECA channel attention module, or a module that only uses global average pooling to generate channel weights; The spatial attention submodule can be replaced by a deformable convolution module; The connection order of the channel attention submodule and the spatial attention submodule can be adjusted.

7. The indoor image fast retrieval method with illumination robustness according to claim 6, characterized in that, The post-processing module includes a global pooling module, a feature projection module, a normalization module, and an L2 normalization module connected in sequence. The global pooling module can be any one of the following: global average pooling module, global max pooling module, or a weighted fusion module of global average pooling and global max pooling. The feature projection module employs a single-layer linear layer or a multi-layer perceptron. The normalization module can be any one of layer normalization module, batch normalization module, or group normalization module.

8. The indoor image retrieval method with illumination robustness according to claim 7, characterized in that, The similarity calculation method is selected from any one of cosine similarity, Euclidean distance, or weighted inner product.

9. The indoor image retrieval method with illumination robustness according to claim 8, characterized in that, The training process of the feature embedding extraction model includes: Obtain a training sample set, wherein the format of the sample images in the training sample set is the same as the format of the standard query image; Data augmentation processing is performed on sample images in the training sample set to generate multiple different augmented views of the same sample image; the data augmentation processing includes color jitter, random rotation, and affine transformation; Multiple augmented views of the same sample image are used as positive sample pairs, and multiple augmented views of different images are used as negative sample pairs. These are input into the feature embedding extraction model to be trained to extract the embedding vectors corresponding to each augmented view. The loss value is calculated using a contrastive learning loss function, which may be either InfoNCE loss or Triplet loss. The model parameters are optimized by backpropagation based on the loss value until the model converges, resulting in a well-trained feature embedding extraction model.

10. The indoor image fast retrieval method with illumination robustness according to claim 1, characterized in that, Before calculating the similarity between the query embedding vector and each vector in the pre-obtained set of reference embedding vectors, a retrieval index is pre-constructed for the set of reference embedding vectors; the retrieval index includes an exact retrieval index or an approximate nearest neighbor retrieval index, the exact retrieval index includes a Flat index, and the approximate nearest neighbor retrieval index includes an HNSW index and an IVF index.

Images