Similar location searching method and device, computer device and storage medium

Abstract

The application discloses a similar place retrieval method and device, computer equipment and a storage medium. The method comprises the following steps: inputting a target place picture into a feature extraction network, extracting global and local feature maps of the target place picture through the feature extraction network; inputting the global and local feature maps into a duplicate checking module, the duplicate checking module retrieving N candidate pictures from a database according to the global and local feature maps, and performing coarse sorting on the N candidate pictures according to a set rule constraint; inputting the target place picture and the candidate pictures after coarse sorting into a fine sorting model, the fine sorting model solving similarity scores between the target place picture and each candidate picture by using a feature relationship model and a geometric feature relationship model, and screening out candidate pictures with similarity scores higher than a set score threshold as similar place retrieval results of the target place picture. The application can greatly improve the retrieval accuracy of similar places.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, computer device, and storage medium for similar location retrieval. Background Technology

[0002] In recent years, with the explosive development of data science and technology, image-based similarity location identification and retrieval, as an important branch of its technical approach, has received widespread attention in both academia and industry.

[0003] In existing technologies, image-based similarity location recognition methods typically use global features for coarse ranking and spatial geometric relationships or relationship networks for fine ranking, without considering the local features of the image, resulting in low accuracy in the final similarity location recognition. Furthermore, since location images usually contain a large amount of interference data unrelated to location retrieval, such as people and vehicles, as well as a large amount of junk data, existing similarity location recognition methods cannot resist the interference of junk data and information, and therefore cannot meet the accuracy requirements of practical applications. Summary of the Invention

[0004] This application provides a similar location retrieval method, apparatus, computer equipment, and storage medium, aiming to solve the technical problem of low similar location identification accuracy in existing similar location identification methods.

[0005] To solve the above-mentioned technical problems, the technical solution adopted in this application is as follows:

[0006] A method for retrieving similar locations includes:

[0007] Retrieve images of the target location to be searched;

[0008] The target location image is input into a trained feature extraction network, which extracts global and local feature maps of the target location image.

[0009] The global and local feature maps are input into the deduplication module. The deduplication module retrieves N candidate images from the database based on the global and local feature maps, and performs a coarse ranking of the N candidate images according to the set rules and constraints to obtain the coarsely ranked candidate images.

[0010] The target location image and the coarsely ranked candidate images are input into the fine ranking model. The fine ranking model uses a feature relationship model and a geometric feature relationship model to solve the similarity score between the target location image and each candidate image, and selects the candidate images with similarity scores higher than a set score threshold as the similar location retrieval results of the target location image.

[0011] The technical solution adopted in this application embodiment also includes: the specific method of obtaining the target location image to be retrieved is as follows:

[0012] The image data of the target location image is obtained through a big data platform. The image data includes the case number, image path, and code of the institution to which the case belongs.

[0013] The target location image to be retrieved is obtained from the IOBS system based on the image data.

[0014] The technical solution adopted in this application embodiment further includes: the feature extraction network includes a global feature extraction module and a local feature extraction module, and the extraction of global and local feature maps of the target location image through the feature extraction network specifically involves:

[0015] The global feature extraction module extracts the global feature map of the target location image through a CNN network or a VIT network. The local feature extraction module extracts the local key information heatmap in the global feature map based on the attention mechanism, and fuses the local key information heatmap with the global feature map to obtain the global and local feature maps F.

[0016] The technical solution adopted in this application embodiment further includes: after extracting the global and local feature maps of the target location image through the feature extraction network, it further includes:

[0017] The global and local feature maps are quality checked using a detection network and a classification network to remove interference information and obtain feature map F'.

[0018] Based on feature map F', determine whether the target location image is an invalid image. If it is an invalid image, return an empty similar location search result for the target location image. If it is not an invalid image, execute the step of inputting the global and local feature maps into the plagiarism detection module.

[0019] The technical solution adopted in this application embodiment further includes: the specific method of using a detection network and a classification network to perform quality detection on the global and local feature maps is as follows:

[0020] The detection network is used to predict invalid regions in the global and local feature maps. The invalid regions are masked, the pixel matrix corresponding to the target region is set to 0, the pixel matrix corresponding to the invalid region is set to 1, and the pixel matrix is ​​multiplied by the global and local feature maps F to obtain the feature map F' after removing interference information.

[0021] Adaptive pooling is performed on the feature map F', and an MLP classification network is used to perform category detection on the target location image to determine whether the target location image is an invalid image; the invalid image refers to an image where the company logo, documents, or location information are severely obscured or the amount of location information is too small.

[0022] The technical solution adopted in this application embodiment further includes: the deduplication module retrieves N candidate images from the database based on global and local feature maps, and performs coarse ranking of the N candidate images according to the set rule constraints, specifically as follows:

[0023] Calculate the similarity between the feature map F' and the feature maps of each historical image in the database, and select N candidate images whose similarity is greater than the set similarity threshold;

[0024] The N candidate images are coarsely ranked according to the set rules and constraints to obtain K candidate images that satisfy the rules and constraints. The specific rules and constraints are: the case-related agency code of the candidate image and the target location image is consistent, and the case-related latitude and longitude distance between the candidate image and the target location image is less than a set threshold.

[0025] The technical solution adopted in this application embodiment further includes: the fine ranking model uses a feature relationship model and a geometric feature relationship model to solve the similarity score between the target location image and each candidate image, specifically:

[0026] The target location image and K candidate images are input into a fine ranking network. The fine ranking network uses CNN or VIT to extract multi-scale features of the target location image and K candidate images. Based on the multi-scale features, a similarity matrix between each pair of pixels of the target location image and each candidate image is calculated. The similarity matrix is ​​encoded using 4D convolution. The relationship score SO between the target location image and each candidate image is predicted through a pooling layer and MLP.

[0027] Calculate the similarity score between each pair of pixels between the target location image and each candidate image. If the similarity score is higher than the set first score threshold, the pixel pairing is considered successful; otherwise, the pixel pairing is considered unsuccessful. This yields m optimal pixel matching pairs. The coordinates of the m optimal pixel matching pairs on the corresponding images are fed into the geometric matching algorithm to calculate the outlier ratio and obtain the spatial geometric score S1.

[0028] The similarity score between the target location image and each candidate image is calculated using the relation score SO and the spatial geometry score S1. If the similarity score is higher than a set second score threshold, the candidate image is used as the similar location retrieval result of the target location image. The similarity score calculation formula is: Score = a*S0 + b*S1, where a and b are hyperparameters.

[0029] Another technical solution adopted in this application embodiment is: a similar location retrieval device, comprising:

[0030] Data acquisition module: used to acquire images of the target location to be searched;

[0031] Feature extraction module: used to input the target location image into the trained feature extraction network, and extract global and local feature maps of the target location image through the feature extraction network;

[0032] Coarse ranking module: This module inputs the global and local feature maps into the deduplication module. The deduplication module retrieves N candidate images from the database based on the global and local feature maps and performs coarse ranking on the N candidate images according to the set rules and constraints to obtain the coarsely ranked candidate images.

[0033] Fine ranking module: It is used to input the target location image and the candidate images after coarse ranking into the fine ranking model. The fine ranking model uses the feature relationship model and the geometric feature relationship model to solve the similarity score between the target location image and each candidate image, and selects the candidate images with similarity scores higher than a set score threshold as the similar location retrieval results of the target location image.

[0034] Another technical solution adopted in this application embodiment is: a computer device, the computer device comprising:

[0035] Memory that stores executable program instructions;

[0036] The processor connected to the memory;

[0037] The processor invokes the executable program instructions stored in the memory to execute the similar location retrieval method as described above.

[0038] Another technical solution adopted in this application embodiment is: a storage medium storing processor-executable program instructions, the program instructions being used to execute the above-mentioned similar location retrieval method.

[0039] The similar location retrieval method, apparatus, computer device, and storage medium of this application extract global and local feature maps of target location images, use these feature maps to retrieve candidate images, and then perform a coarse ranking of the candidate images based on business scenario rule constraints. Finally, they refine the ranking of the candidate images using feature relationship models and geometric feature relationship models to obtain similar location retrieval results for the target location images. This application's embodiment combines business scenario rule constraints for coarse ranking of candidate images with relationship scores and spatial geometric scores for fine ranking, significantly improving the accuracy of similar location retrieval. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the similar location retrieval method according to the first embodiment of this application;

[0041] Figure 2This is a flowchart illustrating the similar location retrieval method according to the second embodiment of this application;

[0042] Figure 3 This is a rough flowchart of the candidate image sorting process for embodiments of this application;

[0043] Figure 4 This is a flowchart illustrating the candidate image ranking process in an embodiment of this application.

[0044] Figure 5 This is a schematic diagram of the structure of the similar location retrieval device according to an embodiment of this application;

[0045] Figure 6 This is a schematic diagram of the computer device structure according to an embodiment of this application;

[0046] Figure 7 This is a schematic diagram of the storage medium structure according to an embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0048] The terms "first," "second," and "third" in this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movements between components in a specific orientation (as shown in the figures). If the specific orientation changes, the directional indications also change accordingly. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0049] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0050] This application's embodiments can acquire and process relevant data based on artificial intelligence (AI) technology. AI is the theory, methods, technology, and application system that uses digital computers or computers-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. Basic AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0051] Please see Figure 1 This is a flowchart illustrating the similar location retrieval method according to the first embodiment of this application. The similar location retrieval method according to the first embodiment of this application includes the following steps:

[0052] S100: Obtain the image of the target location to be searched;

[0053] S110: Input the target location image into the trained feature extraction network, and extract the global and local feature maps of the target location image through the feature extraction network;

[0054] S120: The global and local feature maps are input into the deduplication module. The deduplication module retrieves N candidate images from the database based on the global and local feature maps, and performs a coarse ranking of the N candidate images according to the set rules and constraints to obtain the coarsely ranked candidate images.

[0055] S130: Input the target location image and the candidate images after coarse ranking into the fine ranking model. The fine ranking model uses the feature relationship model and the geometric feature relationship model to solve the similarity score between the target location image and each candidate image, and selects the candidate images with similarity scores higher than a set score threshold as the similar location retrieval results of the target location image.

[0056] Based on the above, the similar location retrieval method of the first embodiment of this application extracts global and local feature maps of the target location image, uses these feature maps to retrieve candidate images, and performs a coarse ranking of the candidate images based on business scenario rule constraints. Then, it refines the ranking of the coarsely ranked candidate images using a feature relationship model and a geometric feature relationship model to obtain the similar location retrieval results for the target location image. This embodiment of the application combines business scenario rule constraints for coarse ranking of candidate images and relationship scores and spatial geometric scores for fine ranking, significantly improving the accuracy of similar location retrieval.

[0057] Please see Figure 2 This is a flowchart illustrating the similar location retrieval method according to the second embodiment of this application. The similar location retrieval method according to the second embodiment of this application includes the following steps:

[0058] S200: Obtain the image of the target location to be searched;

[0059] In this step, the target location image refers to the location of the vehicle accident involved in auto insurance claims. In insurance claims, searching for similar location images of the accident site can serve as a basis for claims personnel to identify abnormal claims. Specifically, image data such as the case number, image path, and agency code of the target location image can be obtained from a big data platform. Based on this image data, the target location image is then retrieved from the IOBS (input / output buffer system).

[0060] S210: Input the acquired target location image into the trained feature extraction network, and extract the global and local feature maps of the target location image through the feature extraction network;

[0061] In this step, the feature extraction network includes a global feature extraction module and a local feature extraction module. The global feature extraction module includes, but is not limited to, CNN (Convolutional Neural Networks) networks or VIT (Vision Transformer, an image classification method based entirely on self-attention mechanisms) networks. The local feature extraction module is a network model based on an attention mechanism. After the global feature extraction module extracts the global feature map of the target location image, the local feature extraction module extracts local key information heatmaps from the global feature map based on the attention mechanism, and then fuses the local key information heatmaps with the global feature map to obtain the global and local feature maps F. The fusion method of the local key information heatmaps and the global feature maps includes, but is not limited to, dot product, direct addition, weighted summation, or orthogonality, etc., and the specific method can be set according to the actual application scenario.

[0062] S220: Use the detection network and classification network to perform quality inspection on the global and local feature maps, remove interference information in the global and local feature maps to obtain feature map F', and determine whether the target location image is an invalid image based on feature map F'. If it is an invalid image, proceed to S230; if it is not an invalid image, continue to S240.

[0063] In this step, since the target location image may contain interfering information unrelated to location retrieval, such as people and vehicles, as well as a large number of invalid images, such as company logos, documents, severely obscured location information, or insufficient location information, in order to reduce the interference of interfering information and invalid images, this embodiment of the application uses a detection network and a classification network to perform quality detection on the global and local feature maps of the target location image, removes interfering information in the global and local feature maps, and identifies invalid images, which helps to improve the retrieval accuracy of subsequent similar location images. Specifically, the quality detection method for global and local feature maps is as follows: First, the detection network predicts invalid regions in the global and local feature maps, masks the invalid regions, sets the pixel matrix corresponding to the target region to 0, and sets the pixel matrix corresponding to the invalid regions other than the target region to 1. Then, the pixel matrix is ​​multiplied by the global and local feature maps F to obtain the feature map F' after removing interference information. Next, adaptive pooling is performed on the feature map F', and an MLP classification network is used to perform category detection on the target location image. The classification network can identify non-location images such as company logos and documents, as well as images with severely obscured location information or insufficient location information. If the target location image is a non-location image such as a company logo or document, or an image with severely obscured location information or insufficient location information, then the target location image is determined to be an invalid image, and the target location image determined to be invalid will not be searched further.

[0064] S230: The search results for similar locations to the target location image are empty;

[0065] S240: Input the feature map after removing interference information into the deduplication module. The deduplication module retrieves N candidate images from the database based on the feature map and performs coarse ranking on the N candidate images according to the set rules and constraints to obtain K candidate images after coarse ranking.

[0066] In this step, such as Figure 3The diagram shown is a flowchart of the candidate image coarse sorting process according to an embodiment of this application. The specific method for candidate image coarse sorting is as follows: First, the similarity between feature map F' and the feature maps of each historical image in the database is calculated, and N candidate images with a similarity greater than a set similarity threshold are selected (if no image has a similarity greater than the set similarity threshold, then N = 0); then, the N candidate images are coarsely sorted according to set rule constraints to obtain candidate images that satisfy both the set similarity threshold and the rule constraints. In this embodiment, the case agency code and other information of the target location image and the candidate images are used as rule constraints. Specifically, the candidate image must be consistent with the case agency code of the target location image, and the case latitude and longitude distance between the candidate image and the target location image must be less than a set threshold.

[0067] Furthermore, the location images and image data of newly generated cases need to be added to the database to update the database. The database involves multiple service instances, and existing retrieval frameworks all require pre-loading the database into memory. However, since each instance receives different data, directly writing the updated data into memory would prevent database updates from being shared among multiple instances. Therefore, this application incorporates a T+t multi-service instance database update module. This module includes M service instances and one service instance that also serves as the database update instance, and allocates a shared space for multiple instances. New location images and their corresponding case numbers, image paths, and other image data are saved to this shared space. Every time t, the module updates the database using the shared space and rewrites the updated database into the shared space. All other instances then reload the updated database, achieving a T+t database update.

[0068] S250: Input the target location image and K candidate images into the fine ranking model. The fine ranking model solves the similarity score between the target location image and each candidate image through the feature relationship model and the geometric feature relationship model, and selects the candidate images with similarity scores higher than the set score threshold as the similar location retrieval results of the target location image.

[0069] In this step, after obtaining the K candidate images after coarse ranking, a fine ranking stage for the candidate images is performed, specifically as follows: Figure 4The diagram shown is a flowchart of the candidate image fine-ranking process according to an embodiment of this application. The candidate image fine-ranking process of this embodiment specifically includes: First, inputting the target location image and K candidate images into the fine-ranking network. The fine-ranking network uses CNN or VIT to extract multi-scale features from the target location image and the K candidate images. Based on the multi-scale features, it calculates the similarity matrix between each pair of pixels between the target location image and each candidate image. The similarity matrix is ​​further encoded using 4D convolution, and then passed through pooling layers and MLP (Multi-layer Pyramid LP). A perceptron (multilayer perceptron) predicts the relationship score SO between the target location image and each candidate image. Then, it calculates the similarity score between each pair of pixels in the target location image and each candidate image. If the similarity score is higher than a set first threshold, the pixel pairing is considered successful; otherwise, if the similarity score is lower than the first threshold, the pixel pairing is considered unsuccessful. This yields m optimal pixel matching pairs. The coordinates of these m optimal pixel matching pairs on the image are then fed into a geometric matching algorithm (RANSAC algorithm) to calculate the outlier ratio, resulting in a spatial geometric score S1. Finally, the similarity score S1 between the target location image and each candidate image is calculated using the relationship score SO and the spatial geometric score S1. The specific formula is: Score = a * S0 + b * S1, where a and b are hyperparameters that can be set according to the actual application scenario. If the similarity score between the target location image and a candidate image is higher than a set second threshold, the candidate image is used as the similarity location retrieval result for the target location image.

[0070] Based on the above, the similar location retrieval method of the second embodiment of this application extracts global and local feature maps of the target location image, performs quality inspection on the global and local feature maps using a detection network and a classification network, removes interference information and invalid data from the global and local feature maps, uses the quality-inspected feature maps to retrieve candidate images, and performs coarse ranking of the candidate images in conjunction with business scenario rule constraints. Finally, it performs fine ranking of the coarsely ranked candidate images using a feature relationship model and a geometric feature relationship model to obtain the similar location retrieval results for the target location image. This embodiment of the application utilizes a detection network and a classification network to perform quality inspection on the global and local feature maps, which can reduce interference from interference information and invalid data, meeting the accuracy requirements of practical applications. Simultaneously, the coarse ranking of candidate images in conjunction with business scenario rule constraints and the fine ranking of candidate images in conjunction with relationship scores and spatial geometric scores greatly improve the accuracy of similar location retrieval.

[0071] In an alternative implementation, the results of the similar location retrieval method can also be uploaded to the blockchain.

[0072] Specifically, based on the results of the similar location retrieval method, corresponding summary information is obtained. Specifically, the summary information is obtained by hashing the results of the similar location retrieval method, for example, using the SHA256s algorithm. Uploading the summary information to the blockchain ensures its security and fairness and transparency to users. Users can download the summary information from the blockchain to verify whether the results of the similar location retrieval method have been tampered with. The blockchain referred to in this example is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. A blockchain is essentially a decentralized database, a chain of data blocks linked using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0073] Please see Figure 5 This is a schematic diagram of the structure of a similar location retrieval device according to an embodiment of this application. The similar location retrieval device 40 according to an embodiment of this application includes:

[0074] Data acquisition module 41: Used to acquire images of the target location to be searched;

[0075] Feature extraction module 42: used to input the target location image into the trained feature extraction network, and extract global and local feature maps of the target location image through the feature extraction network;

[0076] Coarse ranking module 43: used to input the global and local feature maps into the deduplication module. The deduplication module retrieves N candidate images from the database based on the global and local feature maps, and performs coarse ranking on the N candidate images according to the set rules and constraints to obtain the coarsely ranked candidate images.

[0077] Fine ranking module 44: It is used to input the target location image and the candidate images after coarse ranking into the fine ranking model. The fine ranking model uses the feature relationship model and the geometric feature relationship model to solve the similarity score between the target location image and each candidate image, and selects the candidate images with similarity scores higher than a set score threshold as the similar location retrieval results of the target location image.

[0078] The similar location retrieval device in this application extracts global and local feature maps from the target location image, uses these maps to retrieve candidate images, and performs a coarse ranking of the candidate images based on business scenario rules. Then, it refines the ranking of the candidate images using feature relationship models and geometric feature relationship models to obtain the similar location retrieval results for the target location image. This application embodiment combines business scenario rule constraints for coarse ranking of candidate images with relationship scores and spatial geometric scores for fine ranking, significantly improving the accuracy of similar location retrieval.

[0079] Please see Figure 6 This is a schematic diagram of a computer device structure according to an embodiment of this application. The computer device 50 includes:

[0080] Memory 51 storing executable program instructions;

[0081] Processor 52 connected to memory 51;

[0082] The processor 52 is used to call the executable program instructions stored in the memory 51 and perform the following steps: acquire the target location image to be retrieved; input the target location image into a trained feature extraction network, and extract global and local feature maps of the target location image through the feature extraction network; input the global and local feature maps into a deduplication module, which retrieves N candidate images from the database based on the global and local feature maps, and performs coarse ranking on the N candidate images according to set rules to obtain coarsely ranked candidate images; input the target location image and the coarsely ranked candidate images into a fine ranking model, which uses a feature relationship model and a geometric feature relationship model to solve the similarity score between the target location image and each candidate image, and selects candidate images with similarity scores higher than a set score threshold as similar location retrieval results for the target location image.

[0083] The processor 52 can also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor.

[0084] The computer device in this application extracts global and local feature maps from the target location image, uses these maps to retrieve candidate images, and performs a coarse ranking of the candidate images based on business scenario rules. Then, it refines the ranking of the candidate images using a feature relationship model and a geometric feature relationship model to obtain similar location retrieval results for the target location image. This application embodiment combines business scenario rules for coarse ranking of candidate images with relationship scores and spatial geometric scores for fine ranking, significantly improving the accuracy of similar location retrieval.

[0085] Please see Figure 7 , Figure 7 This is a schematic diagram of the storage medium in an embodiment of this application. The storage medium in this embodiment stores program instructions 61 capable of implementing the following steps: acquiring a target location image to be retrieved; inputting the target location image into a trained feature extraction network to extract global and local feature maps of the target location image; inputting the global and local feature maps into a deduplication module, which retrieves N candidate images from a database based on the global and local feature maps, and performs a coarse ranking of the N candidate images according to set rules to obtain coarsely ranked candidate images; inputting the target location image and the coarsely ranked candidate images into a fine ranking model, which uses a feature relationship model and a geometric feature relationship model to solve for the similarity score between the target location image and each candidate image, and selects candidate images with similarity scores higher than a set threshold as similarity location retrieval results for the target location image. The program instructions 61 can be stored in the aforementioned storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in various embodiments of this application. The aforementioned storage media include: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program instructions, or terminal devices such as computers, servers, mobile phones, and tablets. Servers can be independent servers or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0086] The storage medium in this embodiment extracts global and local feature maps of the target location image, uses these maps to retrieve candidate images, and performs a coarse ranking of the candidate images based on business scenario rules. Then, it refines the ranking of the candidate images using feature relationship models and geometric feature relationship models to obtain similar location retrieval results for the target location image. This embodiment combines business scenario rules for coarse ranking of candidate images with relationship scores and spatial geometric scores for fine ranking, significantly improving the accuracy of similar location retrieval.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.

[0088] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for retrieving similar locations, characterized in that, include: Retrieve images of the target location to be searched; The target location image is input into a trained feature extraction network, which extracts global and local feature maps of the target location image. The global and local feature maps are input into the deduplication module. The deduplication module retrieves N candidate images from the database based on the global and local feature maps, and performs a coarse ranking of the N candidate images according to the set rules and constraints to obtain the coarsely ranked candidate images. The target location image and the coarsely ranked candidate images are input into the fine ranking model. The fine ranking model uses a feature relationship model and a geometric feature relationship model to solve the similarity score between the target location image and each candidate image. Specifically, the fine ranking model uses the feature relationship model and the geometric feature relationship model to solve the similarity score between the target location image and each candidate image as follows: The target location image and the coarsely ranked candidate images are input into the fine ranking network. The fine ranking network extracts multi-scale features of the target location image and the coarsely ranked candidate images. Based on the multi-scale features, the similarity matrix between each pair of pixels between the target location image and each candidate image is calculated. The similarity matrix is ​​encoded using 4D convolution. The relationship score between the target location image and each candidate image is obtained by predicting the relationship score between the target location image and each candidate image through a pooling layer and MLP. The similarity score between each pair of pixels between the target location image and each candidate image is calculated. The best pixel matching pair with successful pixel pairing is selected. The coordinates of the best pixel matching pair on the corresponding image are sent to the geometric matching algorithm to calculate the outlier ratio and obtain the spatial geometric score S1. The similarity score between the target location image and each candidate image is calculated using the relationship score S0 and the spatial geometry score S1; and candidate images with similarity scores higher than a set score threshold are selected as similar location retrieval results for the target location image.

2. The similar location retrieval method according to claim 1, characterized in that, The specific steps for obtaining the target location image to be searched are as follows: The image data of the target location image is obtained through a big data platform. The image data includes the case number, image path, and code of the institution to which the case belongs. The target location image to be retrieved is obtained from the IOBS system based on the image data.

3. The similar location retrieval method according to claim 2, characterized in that, The feature extraction network includes a global feature extraction module and a local feature extraction module. The specific steps of extracting global and local feature maps of the target location image through the feature extraction network are as follows: The global feature extraction module extracts the global feature map of the target location image through a CNN network or a VIT network. The local feature extraction module extracts the local key information heatmap in the global feature map based on the attention mechanism, and fuses the local key information heatmap with the global feature map to obtain the global and local feature maps F.

4. The similar location retrieval method according to claim 3, characterized in that, After extracting the global and local feature maps of the target location image through the feature extraction network, the process further includes: The global and local feature maps are quality checked using a detection network and a classification network to remove interference information and obtain feature map F'. Based on feature map F', determine whether the target location image is an invalid image. If it is an invalid image, return an empty similar location search result for the target location image. If it is not an invalid image, execute the step of inputting the global and local feature maps into the deduplication module.

5. The similar location retrieval method according to claim 4, characterized in that, The specific steps of using detection and classification networks to perform quality detection on the global and local feature maps are as follows: The detection network is used to predict invalid regions in the global and local feature maps. The invalid regions are masked, the pixel matrix corresponding to the target region is set to 0, the pixel matrix corresponding to the invalid region is set to 1, and the pixel matrix is ​​multiplied by the global and local feature maps F to obtain the feature map F' after removing interference information. Adaptive pooling is performed on the feature map F', and an MLP classification network is used to perform category detection on the target location image to determine whether the target location image is an invalid image; the invalid image refers to an image where the company logo, documents, or location information are severely obscured or the amount of location information is too small.

6. The similar location retrieval method according to claim 5, characterized in that, The plagiarism detection module retrieves N candidate images from the database based on global and local feature maps, and performs a coarse ranking of the N candidate images according to the set rules and constraints. Calculate the similarity between the feature map F' and the feature maps of each historical image in the database, and select N candidate images whose similarity is greater than the set similarity threshold; The N candidate images are coarsely ranked according to the set rules and constraints to obtain K candidate images that satisfy the rules and constraints. The specific rules and constraints are: the case-related agency code of the candidate image and the target location image is consistent, and the case-related latitude and longitude distance between the candidate image and the target location image is less than a set threshold.

7. The similar location retrieval method according to claim 6, characterized in that, The refined ranking model uses a feature relationship model and a geometric feature relationship model to solve for the similarity score between the target location image and each candidate image, specifically: The target location image and K candidate images are input into a fine ranking network. The fine ranking network uses CNN or VIT to extract multi-scale features of the target location image and K candidate images. Based on the multi-scale features, a similarity matrix between each pair of pixels of the target location image and each candidate image is calculated. The similarity matrix is ​​encoded using 4D convolution. The relationship score SO between the target location image and each candidate image is predicted through a pooling layer and MLP. Calculate the similarity score between each pair of pixels between the target location image and each candidate image. If the similarity score is higher than the set first score threshold, the pixel pairing is considered successful; otherwise, the pixel pairing is considered unsuccessful. This yields m optimal pixel matching pairs. The coordinates of the m optimal pixel matching pairs on the corresponding images are fed into the geometric matching algorithm to calculate the outlier ratio and obtain the spatial geometric score S1. The similarity score between the target location image and each candidate image is calculated using the relation score SO and the spatial geometry score S1. If the similarity score is higher than a set second score threshold, the candidate image is used as the similar location retrieval result for the target location image. The similarity score calculation formula is: Score = a S0+b S1, where a and b are hyperparameters.

8. A similar location retrieval device, characterized in that, include: Data acquisition module: used to acquire images of the target location to be searched; Feature extraction module: used to input the target location image into the trained feature extraction network, and extract global and local feature maps of the target location image through the feature extraction network; Coarse ranking module: This module inputs the global and local feature maps into the deduplication module. The deduplication module retrieves N candidate images from the database based on the global and local feature maps and performs coarse ranking on the N candidate images according to the set rules and constraints to obtain the coarsely ranked candidate images. The fine-ranking module is used to input the target location image and the coarse-ranked candidate images into the fine-ranking model. The fine-ranking model uses a feature relationship model and a geometric feature relationship model to solve the similarity score between the target location image and each candidate image. Specifically, the fine-ranking model inputs the target location image and the coarse-ranked candidate images into the fine-ranking network, extracts multi-scale features of the target location image and the coarse-ranked candidate images using the fine-ranking network, calculates the similarity matrix between each pair of pixels of the target location image and each candidate image based on the multi-scale features, encodes the similarity matrix using 4D convolution, and obtains the relationship score S0 between the target location image and each candidate image through pooling layers and MLP prediction; calculates the similarity score between each pair of pixels of the target location image and each candidate image, selects the best pixel matching pair with successful pixel pairing, and sends the coordinates of the best pixel matching pair on the corresponding image to the geometric matching algorithm to calculate the outlier ratio, obtaining the spatial geometric score S1; The similarity score between the target location image and each candidate image is calculated using the relationship score S0 and the spatial geometry score S1; and candidate images with similarity scores higher than a set score threshold are selected as similar location retrieval results for the target location image.

9. A computer device, characterized in that, The computer device includes: A memory that stores executable program instructions; The processor connected to the memory; The processor invokes the executable program instructions stored in the memory to execute the similar location retrieval method as described in any one of claims 1-7.

10. A storage medium storing processor-executable program instructions, characterized in that, The program instructions are used to execute the similar location retrieval method according to any one of claims 1 to 7.

Images