Image retrieval method and device
By combining object detection networks with global and local feature matching models, the retrieval difficulties caused by image transformation behavior in existing technologies are solved, achieving fast and accurate image retrieval results.
Patent Information
- Application Number
- CN202210001496.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-04
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-01-04
AI Technical Summary
Existing content tracking technologies struggle to handle user actions such as adding emojis, watermarks, cropping, and embedding other photos, making it difficult to quickly and accurately retrieve target images.
A target detection network is used to detect sub-images in the image to be detected. By combining a global feature extraction model and a local feature matching model, and through weighted processing of global feature matching and local feature matching, fast and accurate image retrieval is achieved.
It effectively addresses users' image conversion behavior, enabling rapid and accurate retrieval of target images and improving detection efficiency and accuracy.
Smart Images

Figure CN114547437B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of content tracking, and specifically relates to an image retrieval method and device. Background Art
[0002] Content tracking is a technology widely used on today’s social media platforms, which can be used to flag misinformation, manipulate advertisements, and enforce copyright protection, among other tasks. Specifically, it can quickly respond to viral or urgent offensive content.
[0003] An obvious challenge faced by existing technologies is that the number of photos uploaded to social media platforms every day is usually in the millions or even hundreds of millions. It is impossible to rely entirely on manual review. Therefore, algorithms must be relied upon to help automatically mark or delete inappropriate content.
[0004] In addition, since the uploaded photos may be new attempts by users to sneak past the detection system, such as adding emoticons, adding watermarks, cropping, embedding other photos, etc. Considering the magnitude of the images and the requirements for detection time, how to design an accurate and fast image retrieval solution is a very worthy research issue.
[0005] During the research process, the inventors found that the existing technology currently has at least the following problems: the existing content tracking technology is difficult to deal with users' behaviors such as adding emoticons, adding watermarks, cropping, embedding other photos, etc., which sneak past the detection system, and it is difficult to quickly and accurately retrieve the target image. Summary of the invention
[0006] The purpose of the embodiments of the present application is to provide an image retrieval method and device that can solve the technical problem that the current existing content tracking technology has difficulty dealing with users' behaviors such as adding emoticons, adding watermarks, cropping, embedding other photos, etc., which sneak past the detection system and make it difficult to quickly and accurately retrieve the target image.
[0007] In order to solve the above technical problems, this application is implemented as follows:
[0008] In a first aspect, an embodiment of the present application provides an image retrieval method, comprising:
[0009] Acquire the image to be detected;
[0010] The image to be detected is detected by a target detection network to obtain multiple sub-images to be detected embedded in the image to be detected;
[0011] The global features of each sub-image to be detected are extracted through a global extraction model. The global features can represent various possible forms of the sub-image to be detected after undergoing multiple image transformation methods.
[0012] Through clustering retrieval, multiple similar images matching the global features of the sub-image to be detected are found in the database;
[0013] The local feature matching model is used to perform local feature matching between the sub-image to be detected and the corresponding multiple similar images;
[0014] By weighting the global feature matching results and the local feature matching results, multiple similar images are re-ranked.
[0015] Furthermore, the image retrieval method further includes:
[0016] The test image is obtained by embedding different initial images into the screenshot, or the test image is obtained by embedding an initial image into the screenshot after performing different masking processes;
[0017] The object detection network is trained using the test image and the coordinates of the initial image in the test image.
[0018] Furthermore, the image retrieval method further includes:
[0019] A first set of converted images is obtained by performing a plurality of image conversion methods on the first source image, and a second set of converted images is obtained by performing a plurality of image conversion methods on the second source image, wherein the image conversion methods include: rotation, scaling, cropping, adding filters, adding watermarks, graffiti, stickers or mosaics;
[0020] The global extraction model is trained by using the converted first image set and the second image set.
[0021] Furthermore, local feature matching is performed on the sub-image to be detected and the corresponding multiple similar images through a local feature matching model, specifically including:
[0022] The feature points with the fastest changing rate in the image to be detected and the corresponding multiple similar images are located by using Gaussian convolution algorithm, pyramid multi-resolution algorithm or Gaussian difference algorithm;
[0023] Assign values to the feature directions of the feature points and generate feature descriptors to obtain a set of feature matching points;
[0024] The number of feature matching points is output as the result of local feature matching.
[0025] Furthermore, the global feature matching result is the cosine similarity between the sub-image to be detected and the similar image, and the value range is [-1,1]; the local feature matching result is the number of feature matching points between the sub-image to be detected and the similar image, and the value range is [0,+∞);
[0026] By weighting the global feature matching results and the local feature matching results, specifically:
[0027] The global feature matching results and local feature matching results are weighted by the following formula:
[0028]
[0029] Among them, score is the weighted score, score global is the global feature matching result, score local is the local feature matching result, and α is the weight value.
[0030] In a second aspect, an embodiment of the present application provides an image retrieval device, comprising:
[0031] An acquisition module, used for acquiring an image to be detected;
[0032] A detection module is used to detect the image to be detected through an object detection network to obtain a plurality of sub-images to be detected embedded in the image to be detected;
[0033] An extraction module is used to extract the global features of each sub-image to be detected through a global extraction model. The global features can represent various possible forms of the sub-image to be detected after undergoing multiple image conversion methods;
[0034] A global matching module is used to search for multiple similar images matching the global features of the sub-image to be detected in the database through clustering retrieval;
[0035] A local matching module, used for performing local feature matching between a sub-image to be detected and corresponding multiple similar images through a local feature matching model;
[0036] The weighting module re-sorts multiple similar images by weighting the global feature matching results and the local feature matching results.
[0037] Furthermore, the image retrieval device also includes:
[0038] An embedding module, used to obtain a test image by embedding different initial images into the screenshot, or to obtain a test image by embedding an initial image into the screenshot after performing different masking processes;
[0039] The first training module is used to train the target detection network through the test image and the coordinates of the initial image in the test image.
[0040] Furthermore, the image retrieval device also includes:
[0041] A conversion module, used to obtain a converted first image set by performing a plurality of image conversion methods on the first source image, and obtain a converted second image set by performing a plurality of image conversion methods on the second source image, wherein the image conversion methods include: rotation, scaling, cropping, adding filters, adding watermarks, graffiti, stickers or mosaics;
[0042] The second training module is used to train the global extraction model through the converted first image set and the second image set.
[0043] Furthermore, the local matching module specifically includes:
[0044] A positioning submodule is used to locate the feature points with the fastest changing rate in the image to be detected and the corresponding multiple similar images through Gaussian convolution algorithm, pyramid multi-resolution algorithm or Gaussian difference algorithm;
[0045] The assignment submodule is used to assign values to the feature directions of the feature points and generate feature descriptors to obtain a set of feature matching points;
[0046] The output submodule is used to output the number of feature matching points as the result of local feature matching.
[0047] Furthermore, the global feature matching result is the cosine similarity between the sub-image to be detected and the similar image, and the value range is [-1,1]; the local feature matching result is the number of feature matching points between the sub-image to be detected and the similar image, and the value range is [0,+∞);
[0048] Weighting module, specifically used for:
[0049] The global feature matching results and local feature matching results are weighted by the following formula:
[0050]
[0051] Among them, score is the weighted score, score global is the global feature matching result, score local is the local feature matching result, and α is the weight value.
[0052] In an embodiment of the present application, the target detection image is first used to detect the situations of multiple sub-images to be detected embedded in the image to be detected, and the multiple sub-images to be detected are selected. Afterwards, the global feature extraction model is used to extract global features of each sub-image to be detected after changes in image conversion methods such as adding expressions, adding watermarks, and cropping, and similar images are matched according to the global features. Similar images are then matched with more refined and accurate local features to obtain more accurate retrieval results. The above method can effectively deal with the user's behavior of adding expressions, adding watermarks, cropping, embedding other photos, etc., which sneaks through the detection system, and achieves faster and more accurate retrieval through secondary retrieval of global matching and local matching. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 It is a flowchart of an image retrieval method provided in an embodiment of the present application;
[0054] Figure 2 is a schematic diagram of an output result of a target detection network provided in an embodiment of the present application;
[0055] Figure 3 This is a schematic diagram of an irregular embedding method provided in an embodiment of the present application.
[0056] Figure 4 It is a structural schematic diagram of an image retrieval device provided in an embodiment of the present application.
[0057] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0059] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.
[0060] The image retrieval method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.
[0061] Embodiment 1
[0062] Reference Figure 1 , showing a flow chart of an image retrieval method provided in an embodiment of the present application.
[0063] Image retrieval methods include:
[0064] S101: Acquire an image to be detected.
[0065] The image to be detected may be a screenshot of the content displayed in the electronic device taken by the user.
[0066] Reference Figure 2 , which shows a schematic diagram of the output results of a target detection network provided in an embodiment of the present application.
[0067] in, Figure 2 The video includes a screenshot of what the user has displayed on their electronic device.
[0068] It is understandable that a user's screenshot may often contain multiple sub-images, and searching whether a sub-image contains inappropriate content is a key issue that the embodiments of the present application want to solve.
[0069] If existing content tracking technology is used, the retrieval of such screenshots often fails to obtain effective information.
[0070] S102: Detect the image to be detected by using an object detection network to obtain a plurality of sub-images to be detected embedded in the image to be detected.
[0071] Among them, the object detection network can select multiple small images contained in a large image.
[0072] It can be understood that S102 is a preprocessing of the image to be detected.
[0073] At this time, the retrieval task of a large image can be split into retrieval subtasks of multiple small images. After searching each subtask and then integrating them, the retrieval efficiency can be further improved.
[0074] Reference Figure 2 After detection and selection by the target detection network, the 7 extra sub-images contained in the original screenshot were obtained.
[0075] In practical applications, multiple small images are often embedded in a screenshot after being processed in different ways.
[0076] Among them, the masking process may include irregular embedding process.
[0077] Reference Figure 3 , showing a schematic diagram of an irregular embedding method provided in an embodiment of the present application.
[0078] Irregular embedding processing can include circular border embedding, irregular border embedding, square border embedding, fuzzy embedding, sharp embedding, etc.
[0079] This puts higher requirements on the target detection network.
[0080] Optionally, in order to train the target detection network, the following steps S201 and S202 may be used to complete:
[0081] S201: A test image is obtained by embedding different initial images into a screenshot, or an initial image is subjected to different masking processes and then embedded into a screenshot to obtain a test image.
[0082] Among them, by embedding different initial images into screenshots to obtain test images, the object detection network can be trained to extract sub-images of different styles. By embedding an initial image into a screenshot after performing different masking processes to obtain a test image, the object detection network can be trained to cope with the above irregular embedding process.
[0083] Among them, an initial image is subjected to different masking processes and then embedded into the screenshot to obtain a test image. Specifically, the initial image can be directly multiplied by Figure 3 The test images are obtained using various irregular embedding masks as shown in .
[0084] S202: Train an object detection network using a test image and the coordinates of an initial image in the test image.
[0085] It can be understood that in the test image, the coordinates of each sub-image are known.
[0086] Optionally, a test image pair is input and initial coordinates in the test image are output, and the target detection network is continuously corrected by comparing actual output coordinates with known coordinates to train the target detection network.
[0087] The object detection network trained by the above training method can complete the embedded image detection task more accurately and effectively.
[0088] S103: extracting global features of each sub-image to be detected through a global extraction model. The global features can represent various possible forms of the sub-image to be detected after undergoing multiple image conversion methods.
[0089] It should be noted that the global extraction model can be used to convert each sub-image to be detected into a set of feature vectors, and this feature vector can be used as the global feature of the sub-image to be detected to facilitate subsequent storage and calculation.
[0090] This feature vector can represent various possible forms of the sub-image to be detected after undergoing a variety of image conversion methods. As for the various possible forms after undergoing a variety of image conversion methods, it will be explained below.
[0091] Among the multiple sub-images to be detected (small images) extracted by the target detection network, there is a sub-image with a source image with a high similarity, and the source image can be considered to be the source image of the image to be detected (large image).
[0092] In actual applications, users often perform appropriate image conversion on the source image.
[0093] Conventional image conversion methods include: cropping, rotation, scaling, horizontal flipping, vertical flipping, filters, etc. More complex image conversion methods include: affine transformation, adding watermarks, graffiti, emoticon stickers, etc.; very difficult ones include mosaic, blur, large-scale occlusion, etc.
[0094] The existing technology often trains a network model that can recognize specific image transformations for each transformation, making the neural network expert. This approach has high accuracy, but the time complexity and space complexity are very large. Experiments have shown that it is not advisable to first teach the model to recognize the first transformation, then teach the model to recognize the second transformation, then teach the model to recognize the third transformation, and so on. This order of training models is not advisable, as it will cause the subsequent iterative models to lose the previously learned transformations. All image transformations must be trained at the same time. Therefore, the best way is to allow a neural network to complete the recognition of all image transformation methods.
[0095] Optionally, in order to solve the above problem, the global extraction model can be trained by taking the following steps S301 and S302:
[0096] S301: performing a plurality of image conversion methods on a first source image to obtain a set of converted first images, and performing a plurality of image conversion methods on a second source image to obtain a set of converted second images.
[0097] Among them, the image conversion method includes the various image conversion methods mentioned above, which can specifically be: rotation, scaling, cropping, adding filters, adding watermarks, graffiti, stickers or mosaics.
[0098] It should be noted that the source image can undergo at most N image conversions. When faced with a certain image conversion method, there are two results: one is to take it (with a probability of 50%), and the other is not to take it (with a probability of 50%). For example, when the source image faces the specific image conversion method of adding a watermark, there is a 50% probability of adding a watermark, and there is also a 50% probability of not adding a watermark.
[0099] In this embodiment, taking N equal to 4 as an example, assuming that the source image will undergo 4 image transformations, namely, rotation, cropping, filtering, and watermarking, theoretically, 2 4 There are 4 results, among which only one converted image undergoes all 4 conversions, and only one converted image does not undergo any conversion at all. Most converted images undergo 1-3 image conversions. The detailed conversion results are predictable and will not be described in detail in this embodiment. 4 The converted images are combined into an image collection for subsequent storage and analysis.
[0100] At this time, the global features (i.e., feature vectors) extracted by the global extraction model can represent the various possible forms of the sub-image to be detected after the four image transformation methods (2 4 forms).
[0101] It can be understood that obtaining an image set in the above manner is essentially a data enhancement method.
[0102] S302: Training a global extraction model using the converted first image set and the second image set.
[0103] Optionally, when training with the first image set, the second image set can be used as negative samples for joint training, which can reduce the feature distance caused by image conversion, increase the feature distance between different source images, and make the obtained global extraction model more accurate.
[0104] S104: searching the database for multiple similar images that match the global features of the sub-image to be detected through clustering retrieval.
[0105] Optionally, the cosine similarity global feature matching result of the sub-image to be detected is performed with that of similar images.
[0106] Specifically, by comparing the global feature vector of the source image in the database with the global feature vector of the sub-image to be detected, multiple global feature vectors with the smallest distance to the global feature vector and multiple similar images corresponding thereto are found.
[0107] Optionally, the database uses a Faiss library, which stores a large number of global feature vectors generated from source images.
[0108] It can be understood that cluster retrieval is a preliminary screening process that can quickly determine the most similar images to facilitate subsequent detailed screening.
[0109] S105: Performing local feature matching on the sub-image to be detected and the corresponding multiple similar images through a local feature matching model.
[0110] Optionally, the number of feature matching points between the sub-image to be detected and the similar image is used as the local feature matching result.
[0111] It can be understood that local feature matching is a reprocessing of the multiple similar images obtained in the previous step, and is a further detailed screening of the multiple similar images, which can improve the accuracy of image retrieval.
[0112] Generally speaking, the accuracy of local feature matching algorithms is often higher than that of global feature matching algorithms. However, correspondingly, the speed of local feature matching algorithms is also lower than that of global feature matching algorithms. Therefore, in this embodiment, global feature matching is combined with local feature matching to ensure both retrieval speed and retrieval accuracy.
[0113] Further, S105 can be completed through sub-steps S1051-S1053:
[0114] S1051: Locate the feature points with the fastest changing rate between the image to be detected and the corresponding multiple similar images by using a Gaussian convolution algorithm, a pyramid multi-resolution algorithm or a Gaussian difference algorithm.
[0115] In practical applications, the appropriate local feature matching algorithm can be selected by comprehensively considering hardware conditions, time tolerance, and number of concurrent queries.
[0116] S1052: assigning values to feature directions of feature points to generate feature descriptors to obtain a feature matching point set;
[0117] Here, generating a descriptor specifically refers to describing the feature points through a set of vectors.
[0118] S1053: Output the number of feature matching points as the result of local feature matching.
[0119] It should be noted that the above local feature matching algorithm is completed by autonomous learning of the neural network.
[0120] S106: re-sorting the multiple similar images by weighting the global feature matching results and the local feature matching results.
[0121] Optionally, the image with the highest similarity is displayed first as the first-ranked image (ie, the first image).
[0122] It should be noted that weighted processing refers to assigning different priorities to the global feature matching results and the local feature matching results according to actual conditions to obtain more accurate results.
[0123] It should be noted that the global feature matching result is the cosine similarity between the sub-image to be detected and the similar image, and the value range is [-1,1]. The local feature matching result is the number of feature matching points between the sub-image to be detected and the similar image, and the value range is [0,+∞).
[0124] In order to achieve the weighting of the global feature matching result and the local feature matching result, specifically, the global feature matching result and the local feature matching result can be weighted by the following formula:
[0125]
[0126] Among them, score is the weighted score, score global is the global feature matching result, score local is the local feature matching result, and α is the weight value.
[0127] Optionally, if the local feature algorithm selects a traditional algorithm, the weight of the global feature matching result is higher, and α can be 0.6; if the local feature algorithm selects a neural network algorithm, the weight of the local feature matching result is higher, and α can be 0.4. The specific value of α can be selected according to actual conditions, and the embodiment of the present application is not limited.
[0128] In an embodiment of the present application, the target detection image is used to first detect the situations of multiple sub-images to be detected embedded in the image to be detected, and the multiple sub-images to be detected are selected. Afterwards, the global feature extraction model is used to extract global features of each sub-image to be detected after changes in image conversion methods such as adding expressions, adding watermarks, and cropping, and similar images are matched according to the global features. Similar images are then matched with more refined and accurate local features to obtain more accurate retrieval results. The above method can effectively deal with the user's behavior of adding expressions, adding watermarks, cropping, embedding other photos, etc., which sneaks through the detection system, and achieves faster and more accurate image retrieval through secondary retrieval of global matching and local matching.
[0129] Embodiment 2
[0130] Reference Figure 3 , showing a schematic structural diagram of an image retrieval device provided in an embodiment of the present application.
[0131] The image retrieval device 40 comprises:
[0132] An acquisition module 401 is used to acquire an image to be detected;
[0133] A detection module 402 is used to detect the image to be detected through an object detection network to obtain a plurality of sub-images to be detected embedded in the image to be detected;
[0134] An extraction module 403 is used to extract global features of each sub-image to be detected through a global extraction model, and the global features can represent various possible forms of the sub-image to be detected after undergoing multiple image conversion methods;
[0135] A global matching module 404 is used to search for multiple similar images matching the global features of the sub-image to be detected in the database through clustering retrieval;
[0136] A local matching module 405 is used to perform local feature matching between the sub-image to be detected and the corresponding multiple similar images through a local feature matching model;
[0137] The weighting module 406 re-sorts the multiple similar images by weighting the global feature matching results and the local feature matching results.
[0138] Furthermore, the image retrieval device 40 further includes:
[0139] The embedding module 407 is used to obtain a test image by embedding different initial images into the screenshot, or to obtain a test image by embedding an initial image into the screenshot after performing different masking processes;
[0140] The first training module 408 is used to train the object detection network through the test image and the coordinates of the initial image in the test image.
[0141] Furthermore, the image retrieval device 40 further includes:
[0142] The conversion module 409 is used to obtain a converted first image set by performing a plurality of image conversion methods on the first source image, and obtain a converted second image set by performing a plurality of image conversion methods on the second source image, wherein the image conversion methods include: rotation, scaling, cropping, adding filters, adding watermarks, graffiti, stickers or mosaics;
[0143] The second training module 410 is used to train the global extraction model through the converted first image set and the second image set.
[0144] Furthermore, the local matching module 405 specifically includes:
[0145] The positioning submodule 4051 is used to locate the feature points with the fastest changing rate between the image to be detected and the corresponding multiple similar images by using a Gaussian convolution algorithm, a pyramid multi-resolution algorithm or a Gaussian difference algorithm;
[0146] The assignment submodule 4052 is used to assign values to the feature directions of the feature points and generate feature descriptors to obtain a feature matching point set;
[0147] The output submodule 4053 is used to output the number of feature matching points as the result of local feature matching.
[0148] Furthermore, the global feature matching result is the cosine similarity between the sub-image to be detected and the similar image, and the value range is [-1,1]; the local feature matching result is the number of feature matching points between the sub-image to be detected and the similar image, and the value range is [0,+∞);
[0149] The weighting module 406 is specifically used for:
[0150] The global feature matching results and local feature matching results are weighted by the following formula:
[0151]
[0152] Among them, score is the weighted score, score global is the global feature matching result, score local is the local feature matching result, and α is the weight value.
[0153] The image retrieval device 40 provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.
[0154] In an embodiment of the present application, the target detection image is used to first detect the situations of multiple sub-images to be detected embedded in the image to be detected, and the multiple sub-images to be detected are selected. Afterwards, the global feature extraction model is used to extract global features of each sub-image to be detected after changes in image conversion methods such as adding expressions, adding watermarks, and cropping, and similar images are matched according to the global features. Similar images are then matched with more refined and accurate local features to obtain more accurate retrieval results. The above method can effectively deal with the user's behavior of adding expressions, adding watermarks, cropping, embedding other photos, etc., which sneaks through the detection system, and achieves faster and more accurate image retrieval through secondary retrieval of global matching and local matching.
[0155] The virtual device in the embodiment of the present application may be a device, or a component, an integrated circuit, or a chip in a terminal.
[0156] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims
1. An image retrieval method, characterized in that: include: Acquire the image to be detected; Detecting the image to be detected by a target detection network to obtain a plurality of sub-images to be detected embedded in the image to be detected; Extracting global features of each of the sub-images to be detected by a global extraction model, wherein the global features can represent various possible forms of the sub-images to be detected after undergoing multiple image conversion methods; Searching for multiple similar images matching the global features of the sub-image to be detected in a database through clustering retrieval; Performing local feature matching on the sub-image to be detected and the corresponding multiple similar images through a local feature matching model; Re-ranking the plurality of similar images by weighting the global feature matching results and the local feature matching results; The performing local feature matching on the sub-image to be detected and the corresponding multiple similar images by using a local feature matching model specifically includes: Locating the feature points with the fastest changing rate in the image to be detected and the corresponding multiple similar images by using a Gaussian convolution algorithm, a pyramid multi-resolution algorithm or a Gaussian difference algorithm; Assigning values to the feature directions of the feature points to generate feature descriptors to obtain a feature matching point set; Outputting the number of the feature matching points as the result of the local feature matching; The global feature matching result is the cosine similarity between the sub-image to be detected and the similar image, and the value range is [-1, 1]. The local feature matching result is the number of feature matching points between the sub-image to be detected and the similar image, and the value range is [0, +∞); The weighted processing of the global feature matching result and the local feature matching result is specifically as follows: The global feature matching results and local feature matching results are weighted by the following formula: Among them, score is the weighted score, score global is the global feature matching result, score local is the local feature matching result, and α is the weight value.
2. The method according to claim 1, characterized in that: Also includes: The test image is obtained by embedding different initial images into the screenshot, or the test image is obtained by embedding an initial image into the screenshot after performing different masking processes; The object detection network is trained using the test image and the coordinates of the initial image in the test image.
3. The method according to claim 1, characterized in that Also includes: A first set of converted images is obtained by performing a plurality of image conversion methods on the first source image, and a second set of converted images is obtained by performing a plurality of image conversion methods on the second source image, wherein the image conversion methods include: rotation, scaling, cropping, adding filters, adding watermarks, graffiti, stickers or mosaics; The global extraction model is trained by using the converted first image set and the second image set.
4. An image retrieval device, characterized in that: include: An acquisition module, used for acquiring an image to be detected; A detection module, used to detect the image to be detected through a target detection network to obtain a plurality of sub-images to be detected embedded in the image to be detected; An extraction module, used for extracting global features of each of the sub-images to be detected through a global extraction model, wherein the global features can represent various possible forms of the sub-images to be detected after undergoing multiple image conversion methods; A global matching module, used to search in a database through clustering retrieval for multiple similar images that match the global features of the sub-image to be detected; A local matching module, used for performing local feature matching between the sub-image to be detected and the corresponding multiple similar images through a local feature matching model; A weighting module, which re-sorts the plurality of similar images by weighting the global feature matching results and the local feature matching results; The local matching module specifically includes: A positioning submodule, used to locate the feature points with the fastest changing rate between the image to be detected and the corresponding multiple similar images by using a Gaussian convolution algorithm, a pyramid multi-resolution algorithm or a Gaussian difference algorithm; An assignment submodule, used for assigning values to the feature directions of the feature points, generating feature descriptors, and obtaining a feature matching point set; An output submodule, used for outputting the number of the feature matching points as the result of the local feature matching; The global feature matching result is the cosine similarity between the sub-image to be detected and the similar image, and the value range is [-1, 1]. The local feature matching result is the number of feature matching points between the sub-image to be detected and the similar image, and the value range is [0, +∞); The weighting module is specifically used for: The global feature matching results and local feature matching results are weighted by the following formula: Among them, score is the weighted score, score global is the global feature matching result, score local is the local feature matching result, and α is the weight value.
5. The device according to claim 4, characterized in that Also includes: An embedding module, used to obtain a test image by embedding different initial images into the screenshot, or to obtain a test image by embedding an initial image into the screenshot after performing different masking processes; The first training module is used to train the object detection network through the test image and the coordinates of the initial image in the test image.
6. The device according to claim 4, characterized in that Also includes: A conversion module, used to obtain a converted first image set by performing a plurality of image conversion methods on the first source image, and obtain a converted second image set by performing a plurality of image conversion methods on the second source image, wherein the image conversion methods include: rotation, scaling, cropping, adding filters, adding watermarks, graffiti, stickers or mosaics; The second training module is used to train the global extraction model through the converted first image set and the second image set.
