Continuous image retrieval method and system based on particle filtering and readable storage medium
By adopting particle filtering technology in visual positioning and using the continuous image search method of particle filtering, the problem of fluctuation in image search results is solved, the accuracy of visual map positioning and system robustness are improved, and the image search results are ensured. Stability and efficiency.
Patent Information
- Application Number
- CN202410987455.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-15
AI Technical Summary
During the visual positioning process, the image retrieval results have great fluctuations, which affects the accuracy of visual map positioning. Especially in scenes containing similar textures, incorrect search results often lead to unstable visual positioning results.
Using a continuous image search method based on particle filtering, the keyframe position is extracted from the map sequence image, the particles are initialized and the weight is set, the distance between the particles and the candidate frame is calculated, the particle weight is updated, the image search results are output, and the historical search results are adjusted. Distance and weight improve the stability of the search results.
Through particle filtering technology, the stability of image search results and the accuracy of visual map positioning are improved, mislocalization is reduced, the robustness of the system is enhanced, and the accuracy and efficiency of image search results are ensured.
Smart Images

Figure CN118964662A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a continuous image retrieval method based on particle filtering, a continuous image retrieval system based on particle filtering, and a computer-readable storage medium. Background Art
[0002] In visual map positioning technology, image retrieval technology can be used to deeply analyze the content of visual images, involving the extraction of image features such as image color, texture, shape, etc. After these image features are processed by algorithms, they can be used to efficiently search and match similar images in large databases.
[0003] Image retrieval technology can use certain methods to find the image that is most similar to the image to be retrieved from the existing image library. The specific methods include: obtaining local features from the image and then aggregating them, and directly using deep learning and other methods to obtain global descriptors from the image.
[0004] Local feature re-aggregation methods include BoW (Bag of words) based on traditional manual feature SIFT aggregation, DELG based on local feature aggregation extracted by deep learning, and other algorithms. Among them, deep learning methods that directly obtain the global descriptor of the image include NetVLAD, GeM and other methods.
[0005] Image retrieval technology also involves visual positioning based on keyframes. In the process of visual positioning based on keyframes, the keyframes are first retrieved through the image, and then local matching is performed to obtain the image pose through calculation. Specifically, in the established visual positioning, a mobile device is used to collect certain data within the real space range corresponding to the visual map, and then the pose of the mobile device at the time of triggering positioning is calculated. Each visual positioning is based on the image retrieval of the acquired positioning map, and then the pose of the positioning map in the map is restored through local matching based on the new retrieval results.
[0006] In the prior art, for scenes containing similar textures, the image retrieval results of each visual positioning have large fluctuations, and erroneous retrieval results affect the accuracy of visual map positioning. Summary of the invention
[0007] The technical problem solved by the technical solution of the present invention is: how to improve the stability of image retrieval results during visual positioning, so as to obtain accurate visual map positioning.
[0008] In order to solve the above technical problems, the technical solution of the present invention provides a continuous image retrieval method based on particle filtering, comprising:
[0009] Extracting several key frame positions from the map sequence image;
[0010] Extracting a number of position points based on the key frame positions as positions for initializing particles and setting particle weights;
[0011] Selecting several candidate frames;
[0012] Calculate the distances from all particles to the candidate frame;
[0013] Updating the particle weight according to the distance of each particle to the candidate frame;
[0014] Output the image retrieval results according to the updated particle weights.
[0015] Optionally, extracting a plurality of key frame positions from the map sequence images includes:
[0016] Collect video sequences within the scope of the map scene through a shooting device to establish a visual map;
[0017] Image keyframes of the visual map are extracted to generate the keyframe positions.
[0018] Optionally, extracting a plurality of position points based on the key frame position as positions of initializing particles and setting particle weights includes:
[0019] Extract all keyframe positions;
[0020] Setting a distance threshold and gridding the key frame positions;
[0021] Using the position points retained in each grid as the position of the initialization particle;
[0022] Set initial weights for initial particles.
[0023] Optionally, the selecting a number of candidate frames includes:
[0024] A plurality of candidate frames are retrieved from the key frames based on the current image to be positioned.
[0025] Optionally, let the particle be P i , i is 1, 2, 3, ..., M, M is a natural number greater than 1, and the candidate frame is F j , j is 1, 2, 3, ..., N, N is a natural number greater than 1; particle P i The position coordinate value in the map coordinate system is (X Pi ,Y Pi ,Z Pi ), the candidate frame is F j The position coordinate value in the map coordinate system is (X Fj ,Y Fj ,Z Fj); the calculating the distances of all particles to the candidate frame comprises:
[0026] Set the particle P to be calculated according to the following formula i The position of the candidate key frame F j The distance D of the corresponding position ij :
[0027]
[0028] Statistical Particles P i The position of keyframes F1, F2, F3, ..., F N The corresponding position distance D i1 , D i2 , D i3 , …, D iN The sum of the distances D i :
[0029]
[0030] Get all particles P1, P2, P3, ..., P M To keyframes F1, F2, F3, ..., F N The sum of the distances of the corresponding positions D1, D2, D3, ..., D M ;
[0031] All particles P1, P2, P3, ..., P M The distances to the candidate frames are respectively: the sum of distances D1, D2, D3, ..., D M .
[0032] Optionally, calculating the distances from all particles to the candidate frame further includes:
[0033] When the system has historical search results, adjust the distance sum D according to the location of the historical search results. i To output particles P1, P2, P3, ..., P M To keyframes F1, F2, F3, ..., F N The sum of the corresponding position distances D1', D2', D3', ..., D M ';
[0034] All particles P1, P2, P3, ..., P M The distance to the candidate frame is: the sum of distances D1', D2', D3', ..., D M '.
[0035] Optionally, the distance sum D is adjusted according to the historical search result position i To output particles P1, P2, P3, ..., P MTo keyframes F1, F2, F3, ..., F N The corresponding position distance sum is D1', D2', D3', ..., D M 'include:
[0036] Assume that the historical search result position is C, and the position coordinate value of the historical search result position C in the three-dimensional map coordinate system is (X c ,Y c ,Z c );
[0037] Set the key frame F to be calculated according to the following formula j The position (X Fj ,Y Fj ,Z Fj ) to the position corresponding to the historical search result position C (X c ,Y c ,Z c ) distance E j :
[0038]
[0039] Statistics of each key frame F1, F2, F3, ..., F N The distance from the position of the historical search result position C to the position corresponding to the position E1, E2, E3, ..., E N The mean μ of :
[0040]
[0041] Calculate the distance E j The variance s is:
[0042]
[0043] in, is the mean μ, that is
[0044] Calculate the probability density function G between the historical search results and the current search j :
[0045]
[0046] Based on the probability density function G j Adjust particle P i The position of the candidate key frame F j The distance D of the corresponding position ij To get the adjusted distance D ij ':
[0047]
[0048] Statistical Particles P i to all keyframes F1, F2, F3, ..., F N The corresponding position distance sum D i ':
[0049]
[0050] Get all the adjusted particles P1, P2, P3, ..., P M To keyframes F1, F2, F3, ..., F N The corresponding position distance sum is: D1', D2', D3', ..., D M '.
[0051] Optionally, updating the particle weight according to the distance from each particle to the candidate frame includes:
[0052] The particle weight for this retrieval is obtained based on the distance between all particles and the candidate frame;
[0053] The product of the currently set particle weight and the particle weight during this search is used as the updated particle weight.
[0054] Optionally, let particle P i The distance to the candidate frame is D i0 , the current particle weight is set to W i0 , the particle weight for this search is W i , the updated particle weight is W i ',but:
[0055] W i =1 / D i0
[0056] W i '=W i0 ×W i =W i0 ×(1 / D i0 ).
[0057] Optionally, the continuous image retrieval method based on particle filtering further includes:
[0058] Based on the historical search results, several position points are extracted as the positions of new particles and the weights of the new particles are set.
[0059] Optionally, the continuous image retrieval method based on particle filtering further includes:
[0060] Several position points are randomly extracted near the historical retrieval results as the positions of new particles and the weights of the new particles are set.
[0061] Optionally, the continuous image retrieval method based on particle filtering further includes:
[0062] The particles are resampled according to the updated particle weights.
[0063] Optionally, outputting the image retrieval result according to the updated particle weight includes:
[0064] A weighted average is performed according to the particle weights and particle positions to obtain an image retrieval result.
[0065] Optionally, the step of extracting a plurality of key frame positions from the map sequence images further includes:
[0066] The candidate frames are sorted according to the distance between the image retrieval result and the candidate frames to obtain new key frames.
[0067] In order to solve the above technical problems, the technical solution of the present invention provides a continuous image retrieval system based on particle filtering, comprising:
[0068] An extraction unit, adapted to extract a plurality of key frame positions from a map sequence image;
[0069] A setting unit, adapted to extract a plurality of position points as positions of initialization particles based on the key frame positions and set particle weights;
[0070] A retrieval unit, adapted to select a number of candidate frames;
[0071] a calculation unit, adapted to calculate the distances of all particles to the candidate frame;
[0072] an updating unit, adapted to update the particle weight according to the distance of each particle to the candidate frame;
[0073] The output unit is adapted to output the image retrieval result according to the updated particle weights.
[0074] In order to solve the above technical problems, the technical solution of the present invention provides a continuous image retrieval system based on particle filtering, including: a processor and a memory;
[0075] The memory stores a computer program, and when the processor executes the computer program, the processor performs the steps of the continuous image retrieval method based on particle filtering as described above.
[0076] In order to solve the above technical problems, the technical solution of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the continuous image retrieval method based on particle filtering are implemented as described above.
[0077] The beneficial effects of the technical solution of the present invention include at least:
[0078] The technical solution of the present invention connects multiple data through particle filtering to obtain more stable image retrieval results, thereby improving the accuracy of visual map positioning.
[0079] The technical solution of the present invention can solve the problem of errors in image retrieval results in the visual positioning of the prior art. Since the recall rate of key frames retrieved in the visual map positioning process of the prior art is low in some scenes with a large number of similar structures, the visual positioning results are erroneous and have obvious jumps. The technical solution of the present invention uses continuous images to improve the recall rate of image retrieval based on the continuity of time and space in the visual positioning process, thereby improving the accuracy of positioning and reducing mispositioning.
[0080] The technical solution of the present invention utilizes the continuity of visual map positioning images to improve the retrieval recall rate of subsequent images. In the retrieval solution for improving continuous images based on particle filtering, the particles are first initialized using the map keyframes, and then the weights of the particles are updated after each positioning to obtain the retrieval results, particles with relatively low weights are removed, and new particles are added, and finally a position is calculated based on all particles. According to the position calculated during the particle filtering process, the retrieved keyframes are sorted according to the spatial distance, and the keyframes close to the calculated position are preferentially selected, thereby ensuring the stability of the image retrieval results and further improving the accuracy of visual map positioning.
[0081] In the optional scheme of the technical solution of the present invention, each newly introduced data is observed through particle filtering, and particles with low probability are eliminated, leaving particles with high probability. The technical solution of the present invention uses the historical image retrieval results as observation data. After particle filtering, the particles with large weights are the particles near the correct position, and then a position is calculated based on these particles according to the weighted average. According to the calculated position, the search results are sorted, and the correct image pose is obtained after solving. BRIEF DESCRIPTION OF THE DRAWINGS
[0082] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0083] Figure 1 A schematic diagram of the process flow of a continuous image retrieval method based on particle filtering provided by the technical solution of the present invention;
[0084] Figure 2 A schematic flow chart of another continuous image retrieval method based on particle filtering provided by the technical solution of the present invention;
[0085] Figure 3A schematic flow chart of another method for continuous image retrieval based on particle filtering provided by the technical solution of the present invention;
[0086] Figure 4 A schematic flow chart of another continuous image retrieval method based on particle filtering provided by the technical solution of the present invention;
[0087] Figure 5 A schematic diagram of the structure of a continuous image retrieval system based on particle filtering provided by the technical solution of the present invention;
[0088] Figure 6 A schematic diagram of the structure of another continuous image retrieval system based on particle filtering provided by the technical solution of the present invention. DETAILED DESCRIPTION
[0089] In order to better illustrate the technical solution of the present invention, the present invention is further described below in conjunction with the accompanying drawings.
[0090] In the visual map positioning process of the existing technology, visual image retrieval is a key technical means. Visual image retrieval is divided into two methods: obtaining local features from the image and re-aggregating them, and directly using deep learning and other methods to obtain global descriptors from the image. Local feature re-aggregation methods include BoW (Bag of words) based on traditional manual feature SIFT aggregation, DELG based on local feature aggregation extracted by deep learning and other algorithms. Among them, deep learning methods that directly obtain global descriptors of images include NetVLAD, GeM and other methods.
[0091] However, in the process of visual map positioning in the prior art, each visual image retrieval result is run independently without considering the continuity of the positioning image, which leads to unstable image retrieval results and affects the accuracy of visual map positioning.
[0092] The technical solution of the present invention proposes to use particle filtering technology to deal with the problem of visual map retrieval. In the process of image processing, particle filtering can predict image retrieval results by screening a large number of particles distributed in the state space according to particle weights, and can resample particles based on historical image retrieval results to maintain particle sample diversity. The technical solution of the present invention realizes image retrieval of visual maps through particle filtering technology, which can improve the accuracy and efficiency of image retrieval results and enhance the robustness of image retrieval systems.
[0093] More specifically, the image retrieval process based on particle filtering in the technical solution of the present invention can mainly include the steps of initialization, prediction, update and resampling, and utilizes the advantages of particle filtering in processing nonlinear and non-Gaussian systems to improve the accuracy and efficiency of image retrieval.
[0094] Combination Figure 1 A continuous image retrieval method based on particle filtering is shown, comprising the following steps:
[0095] Step S100: extracting a number of key frame positions from the map sequence images.
[0096] In step S100, the extracting of a plurality of key frame positions from the map sequence image further includes the following processes: capturing a video sequence within the map scene range by a shooting device to establish a visual map; and extracting image key frames of the visual map to generate the key frame positions.
[0097] In this embodiment, a video sequence within a certain spatial range can be collected by a panoramic / fisheye / digital camera and other devices, and a high-precision visual map can be established. When processing map sequence images, multiple extraction methods can be selected or combined according to specific application scenarios and data characteristics.
[0098] For example: for video sequences with frequent content changes, key frames of sequence images can be extracted by using a method based on inter-frame differences or information entropy combined with edge matching rate; for videos with obvious scene transitions, key frames of sequence images can be extracted using a method based on shot boundaries; considering the unique properties that map images may have, such as stable geographic identification and less dynamic changes, custom k-value clustering and content analysis methods can also provide key frames that better meet actual needs. In a specific embodiment, the specific scheme and parameters for extracting key frames can be adjusted according to specific needs to obtain the optimal key frame extraction effect.
[0099] In other embodiments, extracting a number of key frame positions from the map sequence images may further include: sorting the candidate frames according to the distance between the image retrieval result and the candidate frames to obtain new key frames.
[0100] The candidate frames are sorted according to the distance between the image retrieval results and the candidate frames to extract new key frames. The key frame extraction method used can also refer to the distance calculation and similarity comparison of image features, calculate the similarity between the candidate frame features and the retrieval results, etc. to obtain candidate frames based on the image retrieval results.
[0101] Continue to refer Figure 1 , the continuous image retrieval method based on particle filtering of this embodiment also includes:
[0102] Step S101, extracting a number of position points based on the key frame position as the position of the initialization particle and setting the particle weight.
[0103] In step S101, extracting a number of position points based on the key frame position as the position of the initialization particle and setting the particle weight specifically includes the following process:
[0104] Extract all keyframe positions;
[0105] Setting a distance threshold and gridding the key frame positions;
[0106] Using the position points retained in each grid as the position of the initialization particle;
[0107] Set initial weights for initial particles.
[0108] More specifically, in step S101, the particle positions in the image retrieval process need to be initialized and the particle weights need to be set. Based on the collected visual map key frames, the positions of all key frames need to be further extracted, and the extracted key frame positions are gridded by a given distance threshold. The positions retained in each grid are used as the positions of the initialized particles, and each particle is given an initialization weight.
[0109] According to step S101, all particles P = {P1, P2, ... P M}, each particle P i (i is 1, 2, 3, ..., M, M is the particle number and is a natural number greater than 1) can be expressed as: [X Pi ,Y Pi ,Z Pi ,W i0 ]. Among them, W i0 In other embodiments, other schemes can also be used to perform particle initialization on image retrieval:
[0110] Use historical image retrieval results to initialize particles. The individual optimal position (pBest) and global optimal position (gBest) of the initialized particles can be determined with the help of historical image retrieval results (such as the first image retrieval results). Initialize particles by comparing the fitness value of the current position of the particle with the fitness value of the historical optimal position.
[0111] Select or simultaneously randomly generate some particle positions near the historical search results to initialize the particles. The positions of the initialized particles can be set to be randomly generated in the search space near the historical search results to achieve the initialization of the particles.
[0112] Continue to refer Figure 1 , the continuous image retrieval method based on particle filtering of this embodiment also includes:
[0113] Step S102: selecting several candidate frames.
[0114] In this embodiment, when performing image retrieval, some possible candidate frames can be retrieved using a common visual retrieval method according to each positioning map, and these candidate frames are set as F = {F1, F2, ... F N}, each candidate frame is F j (j is 1, 2, 3, ..., N, where N is the candidate frame sequence number and is a natural number greater than 1).
[0115] Common visual retrieval methods include: obtaining local features from images and then aggregating them, and directly using deep learning and other methods to obtain global descriptors from images. Local feature reaggregation methods include BoW (Bag of words) based on traditional manual feature SIFT aggregation, DELG based on local feature aggregation extracted by deep learning, and other algorithms. Among them, deep learning methods that directly obtain global descriptors of images include NetVLAD, GeM and other methods.
[0116] Continue to refer Figure 1 , the continuous image retrieval method based on particle filtering of this embodiment also includes:
[0117] Step S103, calculating the distances from all particles to the candidate frame.
[0118] For all current particles P = {P1, P2, ... P M}, let each particle be P i , i takes 1, 2, 3, ..., M, and the current candidate frame is F = {F1, F2, ... F N}, each candidate frame is F j , j is 1, 2, 3, ..., N; then each particle P i The position coordinate value in the map coordinate system is (X Pi ,Y Pi ,Z Pi ), each candidate frame F j The position coordinate value in the map coordinate system is (X Fj ,Y Fj ,Z Fj ).
[0119] When updating the particle value for the first time, the distances of all particles to the candidate frame are calculated including:
[0120] Set the particle P to be calculated according to the following formula i The position of the candidate key frame F j The distance D of the corresponding position ij :
[0121]
[0122] According to the above formula, when j is 1, 2, 3, ..., N, particle P i The position of keyframes F1, F2, F3, ..., F N The corresponding position distance D i1 , D i2 , D i3 , …, D iN ;
[0123] Distance D i1 , D i2 , D i3 , …, D iN Take the sum of distances D i :
[0124]
[0125] Based on the above process, when i is 1, 2, 3, ..., M, particles P1, P2, P3, ..., P M To keyframes F1, F2, F3, ..., F N The corresponding position distance sums are: D1, D2, D3, ..., D M .
[0126] Therefore, when the particle value is updated for the first time, all particles P1, P2, P3, ..., P M The distance to the candidate frame is: the sum of distances D1, D2, D3, ..., D M .
[0127] In an optional solution of this embodiment, when the particle value is not updated for the first time, the sum of the distances D1, D2, D3, ..., D obtained in the above process of this embodiment can be calculated based on the system history search result. M Adjust to obtain search results that take continuity into account.
[0128] Specifically, calculating the distances from all particles to the candidate frame further includes:
[0129] When the system has historical search results, adjust the distance sum D according to the location of the historical search results. i , that is, the sum of distances D1, D2, D3, ..., D M Adjust to output particles P1, P2, P3, ..., P M To keyframes F1, F2, F3, ..., F N The sum of the distances of the corresponding positions D1', D2', D3', ..., D M ';
[0130] All particles P1, P2, P3, ..., P MThe distance to the candidate frame is: the sum of the adjusted distances D1', D2', D3', ..., D M '.
[0131] More specifically, the distance sum D is adjusted according to the historical search result position. i To output particles P1, P2, P3, ..., P M To keyframes F1, F2, F3, ..., F N The corresponding position distance sum is D1', D2', D3', ..., D M 'include:
[0132] Assume that the historical search result position is C, and the position coordinate value of the historical search result position C in the three-dimensional map coordinate system is (X c ,Y c ,Z c );
[0133] Set the key frame F to be calculated according to the following formula j The position (X Fj ,Y Fj ,Z Fj ) to the position corresponding to the historical search result position C (X c ,Y c ,Z c ) distance E j :
[0134]
[0135] According to the above formula, when j is 1, 2, 3, ..., N, the key frames F1, F2, F3, ..., F N The distances from the positions of the historical search result positions C to the positions corresponding to the positions E1, E2, E3, ..., E N ;
[0136] For position distances E1, E2, E3, ..., E N Take the mean μ:
[0137]
[0138] Calculate the distance E j The variance s of E1, E2, E3, …, E N Calculate the distance variance s:
[0139]
[0140] in, is the mean μ, that is
[0141] When j is 1, 2, 3, ..., N, the probability density functions G1, G2, G3, ..., G4, ..., G5, ..., G6, ..., G7, ..., G8, ..., G9, ..., G10, ..., G11, ..., G12, ..., G13, ..., G14, ..., G15, ..., G16, ..., G17, ..., G18, ..., G19, ..., G20, ..., N :
[0142]
[0143] Based on the probability density function G j Adjust particle P i The position of the candidate key frame F j The distance D of the corresponding position ij To get the adjusted distance D ij ':
[0144]
[0145] According to the above formula, when j is 1, 2, 3, ..., N, adjust each particle P i The position of keyframes F1, F2, F3, ..., F N The corresponding position distance D i1 ', D i2 ', D i3 ', ..., D iN '. In this step, by adjusting each particle P i The position of keyframes F1, F2, F3, ..., F N The sum of the distances of the corresponding positions D i1 , D i2 , D i3 , …, D iN , to adjust the sum of distances D output during the first update i Output the adjusted value of each particle P i The position of keyframes F1, F2, F3, ..., F N The corresponding position distance sum D i1 ', D i2 ', D i3 ', ..., D iN ';
[0146] Based on each particle P i The position of keyframes F1, F2, F3, ..., F N The corresponding position distance D i1 ', D i2 ', D i3 ', ..., D iN ', output particle particle P i To keyframes F1, F2, F3, ..., F N The corresponding position distance D i1 ', D i2 ', Di3 ', ..., D iN The sum of the distances D i ', that is:
[0147]
[0148] Based on the above process, statistics are given for all particles P1, P2, P3, ..., P M To keyframes F1, F2, F3, ..., F N The corresponding position distance sum is D1', D2', D3', ..., D M '.
[0149] Continue to refer Figure 1 , the continuous image retrieval method based on particle filtering of this embodiment also includes:
[0150] Step S104: updating the particle weight according to the distance from each particle to the candidate frame.
[0151] In step S104, updating the particle weight according to the distance from each particle to the candidate frame includes:
[0152] The particle weight for this retrieval is obtained based on the distance between all particles and the candidate frame;
[0153] The product of the currently set particle weight and the particle weight during this search is used as the updated particle weight.
[0154] Specifically, suppose that each particle P i To the candidate frame F j The distance is D i0 ,Right now:
[0155] When updating the particle values for the first time, particles P1, P2, P3, ..., P M The distances to the candidate frames are: D1, D2, D3, ..., D M ;D i0 =D i , i is 1, 2, 3, ..., M;
[0156] When it is not the first time to update the particle value, particles P1, P2, P3, ..., P M The distances to the candidate frames are: D1', D2', D3', ..., D M ';D i0 =D i ', i is 1, 2, 3, ..., M.
[0157] Assume that the current particle P i The initial weight is W i0, the particle weight for this search is W i , the updated particle weight is W i ',but:
[0158] W i =1 / D i0
[0159] W i '=W i0 ×W i =W i0 ×(1 / D i0 ).
[0160] When updating particle values for the first time, there are:
[0161] W i =1 / D i
[0162] W i '=W i0 ×W i =W i0 ×(1 / D i ).
[0163] When updating particle values for a non-first time, there are:
[0164] W i =1 / D i '
[0165] W i '=W i0 ×W i =W i0 ×(1 / D i ').
[0166] Continue to refer Figure 1 , the continuous image retrieval method based on particle filtering of this embodiment also includes:
[0167] Step S105: outputting the image retrieval result according to the updated particle weight.
[0168] After the particle weights are updated, the matching degree between the current position of the particle and the target fitness can be further reflected. After the particle weights are updated, the image retrieval results can be output based on these weights.
[0169] In step S105, outputting the image retrieval result according to the updated particle weight includes: performing weighted averaging according to the particle weight and the particle position to obtain the image retrieval result.
[0170] Specifically, all particles are sorted according to their updated weights. Particles with higher weights indicate that their current positions are closer to the global optimal solution, i.e., the best matching result in image retrieval. The position of the first particle (i.e., the particle with the highest weight) can be regarded as the optimal retrieval result in the current iteration, and the corresponding image can be output as the image retrieval result of this iteration.
[0171] In other embodiments, Figure 2 As shown, the continuous image retrieval method based on particle filtering includes not only the method steps of steps S100 to S105, but also includes:
[0172] Step S106, extracting a number of position points as positions of new particles based on the historical search results and setting the weight of the new particles.
[0173] In the continuous image retrieval method based on particle filtering of the technical solution of the present invention, when using particles for image feature matching and searching, the means of extracting position points as new particles based on the image retrieval results can guide the subsequent search process according to the existing retrieval results, thereby improving the search efficiency and accuracy of the particles. Specifically, the following method can be used to extract several position points as new particle positions based on historical retrieval results and set the weights of the new particles:
[0174] First, it is necessary to determine the retrieved image and its position in the feature space; then, by extracting the position points of the key frame position in a manner similar to step S101, the retrieved image position is divided into grids according to a given distance threshold, and the position retained in each grid is used as the position of the initialization particle, and an initialization weight is assigned to each new particle.
[0175] In other embodiments, Figure 3 As shown, the continuous image retrieval method based on particle filtering includes not only the method steps of steps S100 to S106, but also includes:
[0176] Step S107, randomly extracting several position points near the historical search results as the positions of new particles and setting the weights of the new particles.
[0177] In step S107, by randomly extracting several position points near the known image retrieval results to initialize new particles, the area near the current image retrieval results can be further optimized and explored, thereby increasing the possibility of finding a better image retrieval result. Specifically, several position points can be randomly extracted near the historical retrieval results as the positions of new particles and the weights of new particles can be set in the following manner:
[0178] A search radius is defined near the image retrieval result, and several position points are randomly selected (for example, using a normal distribution or a Gaussian distribution to sample the distribution around the current solution), so as to use the sampled position points as new particle positions, and an initialization weight is given to each new particle.
[0179] In other embodiments, Figure 4 As shown, the continuous image retrieval method based on particle filtering includes not only the method steps of steps S100 to S107, but also includes:
[0180] Step S108, resampling the particles according to the updated particle weights.
[0181] In step S108, the particles of the technical solution of the present invention are resampled by the updated particle weights to increase population diversity, avoid premature convergence, and improve search efficiency. Specifically, the particles can be resampled according to the updated particle weights in the following manner:
[0182] Use the updated particle weights to perform roulette selection (in this selection mechanism, the probability of a particle being selected is proportional to its weight, that is, particles with higher weights are more likely to be selected). Particles with higher weights have a higher probability of being selected, so that more particles with higher weights are retained, while particles with lower weights have a certain chance of survival or are eliminated; or, use random resampling to select a new particle population completely randomly, and the probability of a particle being selected is proportional to its weight.
[0183] In step S108, new particles can also be generated by resampling. According to the result of resampling, low-weight particles can be replaced by high-weight particles, or new particles can be generated near high-weight particles, so as to maintain the diversity of particles and prevent all particles from being too concentrated in a certain local area.
[0184] In other embodiments of the technical solution of the present invention, steps S100 to S105 may be performed in combination with any one or more of steps S106 to S108 to implement the continuous image retrieval method based on particle filtering of the technical solution of the present invention.
[0185] According to the above embodiments, the technical solution of the present invention also provides Figure 5 The continuous image retrieval system based on particle filtering shown includes: an extraction unit, a setting unit, a retrieval unit, a calculation unit, an updating unit and an output unit, which are respectively used to execute steps S100 to S105.
[0186] According to the above embodiments, the technical solution of the present invention also provides Figure 6The continuous image retrieval system based on particle filtering shown includes: a processor and a memory; the memory stores a computer program, and the processor executes the steps of the continuous image retrieval method based on particle filtering in the above embodiment when executing the computer program.
[0187] According to the above embodiment, the technical solution of the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the continuous image retrieval method based on particle filtering described in the above embodiment.
[0188] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.
Claims
1. A continuous image retrieval method based on particle filtering, It is characterized in that include: Extracting several key frame positions from the map sequence image; Extracting a number of position points based on the key frame positions as positions for initializing particles and setting particle weights; Selecting several candidate frames; Calculate the distances from all particles to the candidate frame; Updating the particle weight according to the distance of each particle to the candidate frame; Output the image retrieval results according to the updated particle weights.
2. The continuous image retrieval method based on particle filtering as claimed in claim 1, It is characterized in that The extracting of a plurality of key frame positions from the map sequence images comprises: Collect video sequences within the scope of the map scene through a shooting device to establish a visual map; Image keyframes of the visual map are extracted to generate the keyframe positions.
3. The continuous image retrieval method based on particle filtering as claimed in claim 1, It is characterized in that The step of extracting a plurality of position points based on the key frame position as the position of the initialization particle and setting the particle weight comprises: Extract all keyframe positions; Setting a distance threshold and gridding the key frame positions; Using the position points retained in each grid as the position of the initialization particle; Set initial weights for initial particles.
4. The continuous image retrieval method based on particle filtering as claimed in claim 1, It is characterized in that The selecting of a plurality of candidate frames comprises: A plurality of candidate frames are retrieved from the key frames based on the current image to be positioned.
5. The continuous image retrieval method based on particle filtering as claimed in claim 1, The characteristic is that The particles are P i , i is 1, 2, 3, ..., M, M is a natural number greater than 1, and the candidate frame is F j , j is 1, 2, 3, ..., N, N is a natural number greater than 1; particle P i The position coordinate value in the map coordinate system is (X Pi ,Y Pi , Z Pi ), the candidate frame is F j The position coordinate value in the map coordinate system is (X Fj ,Y Fj ,Z Fj ); the calculating the distances of all particles to the candidate frame comprises: Set the particle P to be calculated according to the following formula i The position of the candidate key frame F j The distance D of the corresponding position ij : Statistical Particles P i The position to key frame F 1 , F 2 , F 3 , …, F N The corresponding position distance D i1 , D i2 , D i3 , …, D iN The sum of the distances D i : Get all particles P 1 , P 2 , P 3 ,…,P M To keyframe F 1 , F 2 , F 3 , …, F N The sum of the distances of the corresponding positions D 1 , D 2 , D 3 , …, D M ; All particles P 1 , P 2 , P 3 ,…,P M The distances to the candidate frames are: the sum of distances D 1 , D 2 , D 3 , …, D M .
6. The continuous image retrieval method based on particle filtering as claimed in claim 5, It is characterized in that The calculating the distances from all particles to the candidate frame further includes: When the system has historical search results, adjust the distance sum D according to the location of the historical search results. i To output particle P 1 , P 2 , P 3 ,…,P M To keyframe F 1 , F 2 , F 3 , …, F N The corresponding position distance sum D 1 ', D 2 ', D 3 ', ..., D M '; All particles P 1 , P 2 , P 3 ,…,P M The distance to the candidate frame is: the sum of distances D 1 ', D 2 ', D 3 ', ..., D M '.
7. The continuous image retrieval method based on particle filtering as claimed in claim 6, It is characterized in that The distance sum D is adjusted according to the historical search result position i To output particle P 1 , P 2 , P 3 ,…,P M To keyframe F 1 , F 2 , F 3 , …, F N The corresponding position distance sum D 1 ', D 2 ', D 3 ', ..., D M 'include: Assume that the historical search result position is C, and the position coordinate value of the historical search result position C in the three-dimensional map coordinate system is (X c ,Y c ,Z c ); Set the key frame F to be calculated according to the following formula j The position (X Fj ,Y Fj ,Z Fj ) to the position corresponding to the historical search result position C (X c ,Y c ,Z c ) j : Statistics of each key frame F 1 , F 2 , F 3 , …, F N The distance E from the location of the historical search result location C 1 , E 2 , E 3 ,…,E N The mean μ of : Calculate the distance E j The variance s is: in, is the mean μ, that is Calculate the probability density function G between the historical search results and the current search j : Based on the probability density function G j Adjust particle P i The position of the candidate key frame F j The distance D of the corresponding position ij To get the adjusted distance Dij': Statistical Particles P i To all keyframes F 1 , F 2 , F 3 , …, F N The corresponding position distance sum D i ': Get all particles P after adjustment 1 , P 2 , P 3 ,…,P M To keyframe F 1 , F 2 , F 3 , …, F N The corresponding position distance sum is: D 1 ', D 2 ', D 3 ', ..., D M '.
8. The continuous image retrieval method based on particle filtering according to any one of claims 1 to 7, It is characterized in that The updating of the particle weight according to the distance of each particle to the candidate frame comprises: The particle weight for this retrieval is obtained based on the distance between all particles and the candidate frame; The product of the currently set particle weight and the particle weight during this search is used as the updated particle weight.
9. The continuous image retrieval method based on particle filtering as claimed in claim 8, The characteristic is that Particle P i The distance to the candidate frame is D i0 , the current particle weight is set to W i0 , the particle weight for this search is W i , the updated particle weight is W i ',but: W i =1 / D i0 W i ’=W i0 ×W i =W i0 ×(1 / D i0 )。 10. The continuous image retrieval method based on particle filtering as claimed in claim 1, It is characterized in that Also includes: Based on the historical search results, several position points are extracted as the positions of new particles and the weights of the new particles are set.
11. The continuous image retrieval method based on particle filtering as claimed in claim 1, It is characterized in that Also includes: Several position points are randomly extracted near the historical retrieval results as the positions of new particles and the weights of the new particles are set.
12. The continuous image retrieval method based on particle filtering as claimed in claim 1, It is characterized in that Also includes: The particles are resampled according to the updated particle weights.
13. The continuous image retrieval method based on particle filtering as claimed in claim 1, It is characterized in that Outputting the image retrieval result according to the updated particle weights includes: A weighted average is performed according to the particle weights and particle positions to obtain an image retrieval result.
14. The continuous image retrieval method based on particle filtering as claimed in claim 13, It is characterized in that The extracting of a plurality of key frame positions from the map sequence images further comprises: The candidate frames are sorted according to the distance between the image retrieval result and the candidate frames to obtain new key frames.
15. A continuous image retrieval system based on particle filtering, It is characterized in that include: An extraction unit, adapted to extract a plurality of key frame positions from a map sequence image; A setting unit, adapted to extract a plurality of position points as positions of initialization particles based on the key frame positions and set particle weights; A retrieval unit, adapted to select a number of candidate frames; a calculation unit, adapted to calculate the distances of all particles to the candidate frame; an updating unit, adapted to update the particle weight according to the distance of each particle to the candidate frame; The output unit is adapted to output the image retrieval result according to the updated particle weights.
16. The continuous image retrieval system based on particle filtering as claimed in claim 15, It is characterized in that The setting unit is also suitable for: Extract all keyframe positions; Setting a distance threshold and gridding the key frame positions; Using the position points retained in each grid as the position of the initialization particle; Set initial weights for initial particles.
17. The continuous image retrieval system based on particle filtering as claimed in claim 15, It is characterized in that The retrieval unit is further adapted to: A plurality of candidate frames are retrieved from the key frames based on the current image to be positioned.
18. The continuous image retrieval system based on particle filtering as claimed in claim 15, The characteristic is that The particles are P i , i is 1, 2, 3, ..., M, M is a natural number greater than 1, and the candidate frame is F j , j is 1, 2, 3, ..., N, N is a natural number greater than 1; particle P i The position coordinate value in the map coordinate system is (X Pi ,Y Pi ,Z Pi ), the candidate frame is F j The position coordinate value in the map coordinate system is (X Fj ,Y Fj ,Z Fj ); the computing unit is also adapted to: Set the particle P to be calculated according to the following formula i The position of the candidate key frame F j The distance D of the corresponding position ij : Statistical Particles P i The position to key frame F 1 , F 2 , F 3 , …, F N The corresponding position distance D i1 , D i2 , D i3 , …, D iN The sum of the distances D i : Get all particles P 1 , P 2 , P 3 ,…,P M To keyframe F 1 , F 2 , F 3 , …, F N The sum of the distances of the corresponding positions D 1 , D 2 , D 3 , …, D M ; All particles P 1 , P 2 , P 3 ,…,P M The distances to the candidate frames are: the sum of distances D 1 , D 2 , D 3 , …, D M .
19. The continuous image retrieval system based on particle filtering as claimed in claim 18, It is characterized in that The computing unit is further adapted to: When the system has historical search results, adjust the distance sum D according to the historical search result position i To output particle P 1 , P 2 , P 3 ,…,P M To keyframe F 1 , F 2 , F 3 , …, F N The corresponding position distance sum D 1 ', D 2 ', D 3 ', ..., D M '; All particles P 1 , P 2 , P 3 ,…,P M The distance to the candidate frame is: the sum of distances D 1 ', D 2 ', D 3 ', ..., D M '.
20. The continuous image retrieval system based on particle filtering as claimed in claim 19, It is characterized in that The computing unit is further adapted to: Assume that the historical search result position is C, and the position coordinate value of the historical search result position C in the three-dimensional map coordinate system is (X c ,Y c ,Z c ); Set the key frame F to be calculated according to the following formula j The position (X Fj ,Y Fj ,Z Fj ) to the position corresponding to the historical search result position C (X c ,Y c ,Z c ) j : Statistics of each key frame F 1 , F 2 , F 3 , …, F N The distance E from the location of the historical search result location C 1 , E 2 , E 3 ,…,E N The mean μ of : Calculate the distance E j The variance s is: in, is the mean μ, that is Calculate the probability density function G between the historical search results and the current search j : Based on the probability density function G j Adjust particle P i The position of the candidate key frame F j The distance D of the corresponding position ij To get the adjusted distance D ij ': Statistical Particles P i To all keyframes F 1 , F 2 , F 3 , …, F N The corresponding position distance sum D i ': Get all particles P after adjustment 1 , P 2 , P 3 ,…,P M To keyframe F 1 , F 2 , F 3 , …, F N The corresponding position distance sum is: D 1 ', D 2 ', D 3 ', ..., D M '.
21. The continuous image retrieval system based on particle filtering according to any one of claims 15 to 20, It is characterized in that The updating unit is further adapted to: The particle weight for this retrieval is obtained based on the distance between all particles and the candidate frame; The product of the currently set particle weight and the particle weight during this search is used as the updated particle weight; let particle P i The distance to the candidate frame is D i0 , the current particle weight is set to W i0 , the particle weight for this search is W i , the updated particle weight is W i ',but: W i =1 / D i0 W i ’=W i0 ×W i =W i0 ×(1 / D i0 )。 22. A continuous image retrieval system based on particle filtering. It is characterized in that include: Processor and memory; The memory stores a computer program, and when the processor executes the computer program, the processor performs the steps of the continuous image retrieval method based on particle filtering as claimed in any one of claims 1 to 14.
23. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the continuous image retrieval method based on particle filtering as claimed in any one of claims 1 to 14 are implemented.
