Image retrieval tag quality inspection methods, devices, electronic equipment, and storage media

Abstract

This disclosure provides an image retrieval tag quality inspection method, apparatus, electronic device, and storage medium. The method includes: determining retrieval tag quality inspection condition information for each target image in a target image sequence; determining reference quality inspection information required for quality inspection of the target retrieval tag information of the target images based on the retrieval tag quality inspection condition information, the reference quality inspection information including at least one of the following: target retrieval tag information of the target image, target location information of the target image, and a first global image feature and a second global image feature extracted from the target image by a global feature extraction model before and after training using the target image's retrieval tag information; and performing quality inspection on the target image's retrieval tag information based on the reference quality inspection information. This disclosure reduces manual quality inspection costs and improves data iteration efficiency in the image retrieval process by introducing reference quality inspection information for rapid and effective quality inspection of image retrieval tag information.

Description

Technical Field

[0001] This disclosure relates to the field of image retrieval technology, and in particular to an image retrieval tag quality inspection method, apparatus, electronic device, and storage medium. Background Technology

[0002] By learning high-dimensional feature representations of images from a large amount of data and constructing an image feature database with localization information, the robot can achieve visual localization based on the high-dimensional features of images during operation.

[0003] In related solutions, the cost of creating image retrieval datasets is relatively high, and the quality of image retrieval labels is difficult to guarantee. For example, it is difficult to identify salient scenes in indoor scenes, and there are also a large number of similar scenes, which makes image annotation difficult and it is hard to obtain high-quality annotations. Therefore, it is particularly important to explore how to automatically generate and annotate the quality of corresponding image retrieval datasets in the actual implementation process. Summary of the Invention

[0004] This disclosure provides an image retrieval tag quality inspection method, apparatus, electronic device, and storage medium to achieve rapid and effective quality inspection of image retrieval tag information, which helps reduce the cost of manual quality inspection and improve the data iteration efficiency in the image retrieval process.

[0005] In a first aspect, embodiments of this disclosure provide an image retrieval tag quality inspection method, the method comprising:

[0006] Determine the retrieval tag quality inspection condition information for each target image in the target image sequence;

[0007] Based on the retrieval tag quality inspection condition information, the reference quality inspection information required for quality inspection of the target retrieval tag information of the target image is determined. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image, respectively.

[0008] The retrieval tag information of the target image is inspected based on the reference quality inspection information.

[0009] Secondly, embodiments of this disclosure also provide an image retrieval tag quality inspection device, the device comprising:

[0010] The retrieval tag quality inspection condition information determination module is used to determine the retrieval tag quality inspection condition information for each target image in the target image sequence;

[0011] The reference quality inspection information determination module is used to determine the reference quality inspection information required when inspecting the target retrieval tag information of the target image based on the retrieval tag quality inspection condition information. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image, respectively.

[0012] The tag information quality inspection module is used to perform quality inspection on the tag information of the target image based on the reference quality inspection information.

[0013] Thirdly, this disclosure also provides an image retrieval tag quality inspection electronic device, the electronic device comprising:

[0014] One or more processors;

[0015] Storage device for storing one or more programs.

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the image retrieval tag quality inspection method according to any embodiment of this disclosure.

[0017] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image retrieval tag quality inspection method described in any embodiment of this disclosure.

[0018] This disclosure provides an image retrieval tag quality inspection method. It determines the retrieval tag quality inspection condition information for each target image in a target image sequence; based on the retrieval tag quality inspection condition information, it determines reference quality inspection information needed for quality inspection of the target retrieval tag information of the target image. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target location information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the target image's retrieval tag information; and it performs quality inspection on the target image's retrieval tag information based on the reference quality inspection information. This disclosure method, by introducing reference quality inspection information to quickly and effectively inspect the retrieval tag information of the target image, verifies whether the annotation quality of the retrieval tag information meets the requirements of image retrieval, helps reduce manual quality inspection costs, further accelerates the data iteration process in image retrieval, and improves data iteration efficiency.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0021] Figure 1 This is a flowchart of an image retrieval tag quality inspection method provided in an embodiment of this disclosure;

[0022] Figure 2 This is a flowchart of another image retrieval tag quality inspection method provided in this disclosure embodiment;

[0023] Figure 3 This is a flowchart of yet another image retrieval tag quality inspection method provided in this disclosure embodiment;

[0024] Figure 4 This is a flowchart of yet another image retrieval tag quality inspection method provided in this disclosure embodiment;

[0025] Figure 5 This is a schematic diagram of a mean set matching operation provided in an embodiment of this disclosure;

[0026] Figure 6 This is a flowchart of yet another image retrieval tag quality inspection method provided in this disclosure embodiment;

[0027] Figure 7 This is a schematic diagram of the structure of an image retrieval tag quality inspection device provided in an embodiment of this disclosure;

[0028] Figure 8 This is a schematic diagram of the structure of an image retrieval tag quality inspection electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0029] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0030] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0031] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0032] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0033] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0034] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0035] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0036] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0037] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0038] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0039] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0040] Figure 1 This is a flowchart illustrating an image retrieval tag quality inspection method provided in this embodiment. This embodiment is applicable to situations requiring rapid and effective quality inspection of image retrieval tags. The method can be executed by an image retrieval tag quality inspection device, which can be implemented in software and / or hardware, optionally through an electronic device such as a mobile terminal, PC, or server. Figure 1 As shown, the image retrieval tag quality inspection method provided in this embodiment may include the following steps:

[0041] S110. Determine the retrieval tag quality inspection condition information for each target image in the target image sequence.

[0042] The target image can refer to an image with search tags awaiting quality inspection. The search tags serve as the basis for image retrieval. For example, search tags may include category tags, which can be used to characterize the category to which an image belongs. The target image sequence can refer to an image sequence composed of multiple target images. Search tag quality inspection condition information can refer to the prerequisites for quality inspection of the search tags of the target images.

[0043] As an optional but non-limiting implementation, the retrieval tag quality inspection condition information includes the retrieval tag usage scenario and the retrieval tag quality inspection method. The retrieval tag usage scenario includes using the global image features of the target image for coarse location on a known map and using the global image features of the target image to determine whether a preset scenario has been reached. The retrieval tag quality inspection method is used to indicate the quality inspection process for the target retrieval tag information of the target image under the retrieval tag usage scenario.

[0044] The search tag usage scenario can be used to characterize the application scenarios of image search tags. Specifically, search tag usage scenarios can include two categories: the first is to use the global image features of the target image for coarse geographical location on a known map; the second is to use the global image features of the target image to determine whether a preset scenario has been reached. The global image features can refer to the overall attributes of the image, specifically including color features, line features, texture features, and structural features. Known maps can include maps such as Amap or Baidu Maps. The preset scenario can refer to a pre-defined image search application scenario. For example, preset scenarios can include application scenarios such as warehousing, handling, and indoor robot movement.

[0045] The retrieval tag quality inspection method can be used to indicate the quality inspection process for target retrieval tag information of a target image in a retrieval tag usage scenario. Target retrieval tag information refers to the retrieval tag information of the target image. For example, target retrieval tag information may include category tag information. The quality inspection process can be understood as the detailed steps for quality inspection of the retrieval tag information of the target image.

[0046] In this embodiment, the usage scenario of the retrieval tags for each target image in the target image sequence is first determined, and then the corresponding retrieval tag quality inspection method is determined based on the retrieval tag usage scenario. It should be noted that different retrieval tag usage scenarios correspond to different retrieval tag quality inspection methods.

[0047] By using the above method, the usage scenarios and quality inspection methods of the retrieved tags can be clearly identified through the information on the tag quality inspection conditions. This allows for better adaptation to application scenarios and quality inspection methods in the future, thereby improving the accuracy and effectiveness of the retrieved tag quality inspection.

[0048] S120. Based on the retrieval tag quality inspection condition information, determine the reference quality inspection information required when performing quality inspection on the target retrieval tag information of the target image. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image.

[0049] The reference quality inspection information can serve as a basis for quality inspection of the target retrieval label information. Specifically, the reference quality inspection information may include at least one of the following: the target retrieval label information of the target image, the target location information of the target image, and the first and second global image features extracted from the target image by the global feature extraction model before and after training using the target image's retrieval label information. The target location information can refer to the location information of the target image. The global feature extraction model can refer to a network model (e.g., a deep learning model) capable of performing global feature extraction on an image. The first and second global image features can refer to the global image features extracted from the target image using the global feature extraction model before and after training, respectively.

[0050] In this embodiment, after determining the retrieval tag quality inspection condition information, reference quality inspection information required for quality inspection of the target retrieval tag information of the target image can be further determined based on the retrieval tag quality inspection condition information. For example, the reference quality inspection information for the first type of retrieval tag usage scenario may include the target retrieval tag information of the target image, target positioning information, a first global image feature, and a second global image feature; the reference quality inspection information for the second type of retrieval tag usage scenario may include the target retrieval tag information of the target image, a first global image feature, and a second global image feature.

[0051] S130. Perform quality inspection on the retrieval tag information of the target image based on the reference quality inspection information.

[0052] In this embodiment, after determining the reference quality inspection information, the retrieval tag information of the target image can be further inspected based on the reference quality inspection information. For example, a coordinate scatter plot can be drawn for each target image in the target image sequence based on the target image's target location information. Target images with the same target retrieval tag information are assigned the same color, while target images with different target retrieval tag information are assigned different colors. If the same color is concentrated in the coordinate scatter plot, it indicates that the labeling quality of the retrieval tag information is good.

[0053] The technical solution of this disclosure involves determining the retrieval tag quality inspection condition information for each target image in a target image sequence; determining reference quality inspection information required for quality inspection of the target retrieval tag information of the target image based on the retrieval tag quality inspection condition information, wherein the reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image; and performing quality inspection on the retrieval tag information of the target image based on the reference quality inspection information. By introducing reference quality inspection information to quickly and effectively inspect the retrieval tag information of the target image, this technical solution verifies whether the annotation quality of the retrieval tag information meets the requirements of image retrieval, helps reduce the cost of manual quality inspection, further accelerates the data iteration process in the image retrieval process, and improves data iteration efficiency.

[0054] Figure 2 This is a flowchart illustrating another image retrieval tag quality inspection method provided in this disclosure. This disclosure further optimizes the aforementioned embodiments, and can be combined with various optional solutions from one or more of the above embodiments. Figure 2 As shown, the image retrieval tag quality inspection method provided in this embodiment may include the following steps:

[0055] S210. Determine the retrieval tag quality inspection condition information for each target image in the target image sequence.

[0056] S220. Based on the retrieval tag quality inspection condition information, determine the reference quality inspection information required when inspecting the target retrieval tag information of the target image. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image.

[0057] S230. Generate target scatter plot information based on the target location information included in each target image. The target location is used to describe the horizontal and vertical coordinates of the image acquisition location of the target image. Each scatter point in the target scatter plot information corresponds to a target image.

[0058] The target location can refer to the positional information when the target image was acquired, specifically the horizontal and vertical coordinates describing the image acquisition location. The target scatter plot information refers to the scatter plot generated based on the target location. In this scatter plot, each point corresponds to one target image.

[0059] In this embodiment, the target position in the target localization information of each target image is first determined. For example, SLAM (Simultaneous Localization and Mapping) and IMU (Inertial Measurement Unit) can be used to obtain the planar coordinate information of the camera relative to a preset origin within the building when acquiring each target image, thereby determining the target position information corresponding to each target image. The preset origin can refer to a coordinate origin pre-set within the building, which can serve as a reference for determining the target position corresponding to the target image. It should be noted that this embodiment does not impose any limitations on the specific location of the preset origin and can be flexibly set according to actual needs. Then, for each target image in the target image sequence, target scatter plot information is drawn based on the target position corresponding to each target image. In the target scatter plot information, each scatter point corresponds to one target image.

[0060] S240. Based on the target retrieval label information, the first global image features, and the second global image features of each target image, the scattered points in the target scatter plot information are marked. The same scattered point in the target scatter plot information is marked using different marking operations. Each dimension of the data in the reference quality inspection information corresponds to a marking operation.

[0061] The marking operation can include operations such as marking colors or marking characters. In this embodiment, the scatter points in the target scatter plot information can be marked from three dimensions: target retrieval label information of each target image, first global image features, and second global image features. Different marking operations are used to mark the same scatter point in the target scatter plot information, and each dimension of the reference quality inspection information corresponds to one marking operation; that is, different dimensions of the reference quality inspection information correspond to different marking operations.

[0062] For example, using color marking as the marking operation, firstly, based on the target retrieval label information of each target image, the scatter points in the target scatter plot are color-marked. Then, the pre-trained global feature extraction model is used to extract the corresponding first global image features from each target image, and the scatter points in the target scatter plot are color-marked based on the first global image features. Next, the trained global feature extraction model is used to extract the corresponding second global image features from each target image, and the scatter points in the target scatter plot are color-marked based on the second global image features. Specifically, for each dimension of reference quality inspection information, target images with the same reference quality inspection information are assigned the same color, and target images with different reference quality inspection information are assigned different colors.

[0063] As an optional but non-limiting implementation, the scatter points in the target scatter plot information are marked based on the target retrieval label information, the first global image features, and the second global image features of each target image, including but not limited to steps A1-A3:

[0064] Step A1: Based on the target category label included in the target retrieval label information of the target image, perform a first labeling operation on the scatter points corresponding to the target image in the target scatter plot information. The label value of the first labeling operation matches the target category label of the target image.

[0065] Here, the target category label can refer to the category label of the target image, which can be used to indicate the category of the target image. The first labeling operation can refer to the labeling operation performed based on the target category label. Specifically, the label value of the first labeling operation matches the target category label of the target image. That is, target images with the same target category label have the same label value, while target images with different target category labels have different label values.

[0066] Step A2: Determine the first reference category label of each target image based on the first global image features of each target image, and perform a second labeling operation on the scatter points corresponding to the target images in the target scatter plot information based on the first reference category label of the target images. The label value of the second labeling operation matches the first reference category label of the target images.

[0067] The first reference category label can be a reference category label determined based on the first global image features, which can be used to represent the category information corresponding to the first global image features. The second labeling operation can be a labeling operation performed based on the first reference category label. Specifically, the label value of the second labeling operation matches the first reference category label of the target image. That is, target images with the same first reference category label have the same label value, while target images with different first reference category labels have different label values.

[0068] Step A3: Determine the second reference category label of each target image based on the second global image features of each target image, and perform a third labeling operation on the scatter points corresponding to the target images in the target scatter plot information based on the second reference category label of the target images. The label value of the third labeling operation matches the second reference category label of the target images.

[0069] The second reference category label can refer to a reference category label determined based on the second global image features, which can be used to characterize the category information corresponding to the second global image features. The third labeling operation can refer to a labeling operation performed based on the second reference category label. Specifically, the label value of the third labeling operation matches the second reference category label of the target image. That is, target images with the same second reference category label have the same label value, while target images with different second reference category labels have different label values.

[0070] Using the above method, different labeling operations can be performed on the scatter points corresponding to the target image in the target scatter plot information based on reference quality inspection information of different dimensions, so as to provide an accurate basis for subsequent quality inspection of retrieval label information.

[0071] As an optional but non-limiting implementation, determining the first reference category label for each target image based on the first global image features of each target image may include, but is not limited to, the following process:

[0072] Based on the first global image features of each target image, a first feature clustering analysis is performed on each target image, and target images in the same cluster are assigned the same first reference category label.

[0073] The first feature clustering analysis can refer to performing clustering analysis on each target image based on a first global image feature. In this embodiment, after determining the first global image feature of each target image, clustering analysis can be performed on each target image based on the first global image feature, and target images belonging to the same cluster can be assigned the same first reference category label.

[0074] As an optional but non-limiting implementation, determining the second reference category label of each target image based on the second global image features of each target image may include, but is not limited to, the following process:

[0075] Based on the second global image features of each target image, a second feature clustering analysis is performed on each target image, and target images in the same cluster are assigned the same second reference category label.

[0076] The second feature clustering analysis can refer to performing clustering analysis on each target image based on the second global image features. In this embodiment, after determining the second global image features of each target image, clustering analysis can be performed on each target image based on the second global image features, and target images belonging to the same cluster can be assigned the same second reference category label.

[0077] It should be noted that this embodiment does not impose any limitations on the clustering analysis methods used for the first feature clustering analysis and the second feature clustering analysis, and can be flexibly set according to actual needs. For example, the DBSCAN (Density-Based Noise-Based Spatial Clustering) method can be used.

[0078] Using the above method, cluster analysis can be performed on each target image based on the first global image features and the second global image features respectively. Then, based on the cluster analysis results, the target images are assigned corresponding reference category labels so that the scatter points in the target scatter plot information can be marked according to the reference category labels.

[0079] S250. Based on the label values ​​obtained by applying different labeling operations to each scatter point in the target scatter plot information, perform quality inspection on the retrieval label information of the target image.

[0080] As an optional but non-limiting implementation, the retrieval label information of the target image is quality checked based on the label values ​​obtained by applying different labeling operations to each scatter point in the target scatter plot information, including but not limited to steps B1-B2:

[0081] Step B1: Determine the positional distribution of scatter points with the same label value in the target scatter plot information when any labeling operation is used for scatter point labeling.

[0082] In this embodiment, for each marking operation, the positional distribution of each scatter point with the same marking value in the target scatter plot information is determined in the target scatter plot.

[0083] Step B2: If the scattered points with the same label value are concentrated in a preset size and position area, then the labeling quality of the target image's retrieval label information is determined to meet the preset quality conditions.

[0084] The preset size and location area can refer to a fixed-size location area set in advance. The preset quality condition can refer to a preset annotation quality condition. In this embodiment, after determining the location distribution of each scatter point with the same label value for each labeling operation in the target scatter plot information, the annotation quality can be judged to meet the preset quality condition based on the location distribution. Specifically, if the scatter points with the same label value are concentrated in the preset size and location area, it can be determined that the annotation quality of the retrieval label information of the target image meets the preset quality condition, indicating that the annotation quality of the retrieval label information is good; conversely, it can be determined that the annotation quality of the retrieval label information of the target image does not meet the preset quality condition, indicating that the annotation quality of the retrieval label information is poor.

[0085] Using the above method, we can quickly and accurately determine whether the labeling quality of the target image's retrieval label information meets the preset quality conditions based on the positional distribution of each scatter point with the same label value in the target scatter plot.

[0086] As an optional but non-limiting implementation, the retrieval label information of the target image is quality checked based on the label values ​​obtained by applying different labeling operations to each scatter point in the target scatter plot information, including but not limited to steps C1-C4:

[0087] Step C1: Determine the first position distribution of each scatter point with different label values ​​in the target scatter plot information when the first labeling operation is used for scatter point labeling.

[0088] The first position distribution can refer to the position distribution of each scatter point with different label values ​​in the target scatter plot information when the first labeling operation is used to mark the scatter points.

[0089] Step C2: Determine the second position distribution of each scatter point with different label values ​​in the target scatter plot information when the second labeling operation is used for scatter point labeling.

[0090] The second position distribution can refer to the position distribution of each scatter point with different label values ​​in the target scatter plot information when the second labeling operation is used to mark the scatter points.

[0091] Step C3: Determine the third position distribution of each scatter point with different label values ​​in the target scatter plot information when the third label operation is used for scatter point labeling.

[0092] The third position distribution can refer to the position distribution of each scatter point with different label values ​​in the target scatter plot information when the third labeling operation is used to mark the scatter points.

[0093] Step C4: Based on the overlap between the first and second position distributions of the same label value and the overlap between the first and third position distributions of the same label value, determine the annotation quality of the retrieval label information of the target image. The higher the overlap between the two position distributions, the higher the annotation quality of the retrieval label information of the target image.

[0094] In this embodiment, after determining the first, second, and third positional distributions, the overlap between the first and second positional distributions with the same label value, as well as the overlap between the first and third positional distributions with the same label value, can be calculated. The annotation quality of the target image's retrieval label information is then determined based on the overlap. Specifically, the higher the overlap between two positional distributions, the higher the annotation quality of the target image's retrieval label information.

[0095] Using the above method, the labeling quality of the retrieval label information of the target image can be quickly and accurately determined based on the overlap between the positional distributions corresponding to two different label operations under the same label value.

[0096] The technical solution of this disclosure generates target scatter plot information based on the target location included in the target positioning information of each target image. The target location is used to describe the horizontal and vertical coordinates of the image acquisition location of the target image. Each scatter point in the target scatter plot information corresponds to one target image. The scatter points in the target scatter plot information are marked based on the target retrieval tag information, the first global image feature, and the second global image feature of each target image. Different marking operations are used to mark the same scatter point in the target scatter plot information. Each dimension of the data in the reference quality inspection information corresponds to a marking operation. The retrieval tag information of the target image is quality inspected based on the mark values ​​obtained by applying different marking operations to each scatter point in the target scatter plot information. By adopting the technical solution of this disclosure, the retrieval tag information of the target image is quickly and effectively inspected by introducing reference quality inspection information to verify whether the annotation quality of the retrieval tag information meets the requirements of image retrieval. This helps reduce the cost of manual quality inspection, further accelerates the data iteration process in the image retrieval process, and improves the data iteration efficiency. Furthermore, the accuracy and efficiency of the retrieval tag information quality inspection are further improved by using the mark values ​​under different marking operations to inspect the retrieval tag information of the target image.

[0097] Figure 3 This is a flowchart illustrating yet another image retrieval tag quality inspection method provided in this disclosure. This disclosure further optimizes the aforementioned embodiments, and can be combined with various optional solutions from one or more of the above embodiments. Figure 3 As shown, the image retrieval tag quality inspection method provided in this embodiment may include the following steps:

[0098] S310. Determine the retrieval tag quality inspection condition information for each target image in the target image sequence.

[0099] S320. Based on the retrieval tag quality inspection condition information, determine the reference quality inspection information required when inspecting the target retrieval tag information of the target image. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image.

[0100] S330. Generate difference scatter plot information based on the target positioning information of each target image. Each scatter point in the difference scatter plot information corresponds to a pair of target images. Each scatter point in the difference scatter plot information is determined by the positional and angular differences between a pair of target images.

[0101] The difference scatter plot information can be used to characterize the positional and angular differences between two target images. Specifically, each scatter point in the difference scatter plot information corresponds to a pair of target images, and each scatter point in the difference scatter plot information can be determined by the positional and angular differences between a pair of target images.

[0102] As an optional but non-limiting implementation, difference scatter plot information is generated based on the target localization information of each target image, including but not limited to steps D1-D2:

[0103] Step D1: Based on the target position and target angle included in the target positioning information of each target image, determine the absolute value of the position difference and the absolute value of the angle difference between any two target images.

[0104] Here, the target angle refers to the angle information when the target image was acquired. The absolute value of the position difference can be used to characterize the positional difference between two target images. The absolute value of the angle difference can be used to characterize the angular difference between two target images.

[0105] In this embodiment, the target position and target angle in the target localization information of each target image are first determined. For example, the target position and target angle corresponding to each target image can be determined by SLAM (Simultaneous Localization and Mapping) and IMU (Inertial Measurement Unit). Then, the absolute value of the position difference and the absolute value of the angle difference between any two target images are calculated.

[0106] Step D2: Construct a difference scatter plot using the absolute values ​​of the positional differences and angle differences between each pair of target images as the x and y coordinates. The target position is used to describe the x and y coordinates of the image acquisition position of the target image, and the target angle describes the acquisition direction when acquiring the target image.

[0107] In this embodiment, after determining the absolute value of the positional difference and the absolute value of the angle difference between each pair of target images, a scatter plot of difference information can be drawn, with the absolute value of the positional difference between each pair of target images as the abscissa and the absolute value of the angle difference between each pair of target images as the ordinate. Here, the target position describes the abscissa and ordinate of the image acquisition position of the target image, and the target angle describes the acquisition direction when acquiring the target image.

[0108] Using the above method, a difference scatter plot is constructed with the absolute values ​​of the positional difference and the absolute values ​​of the angle difference between each pair of target images as the horizontal and vertical axes. This clearly and intuitively reflects the positional and angle differences between the two target images.

[0109] S340. Based on the first global image features and the second global image features of each target image, the scatter points in the difference scatter plot information are marked. The value of the scatter point is determined by the similarity of the global image features between the pair of target images corresponding to the scatter point.

[0110] Global image feature similarity can be used to characterize the degree of similarity of global image features between two target images. It is understood that global image feature similarity is positively correlated with the similarity of target images; that is, the higher the global image feature similarity, the higher the similarity between the two target images; conversely, the lower the global image feature similarity, the lower the similarity between the two target images.

[0111] As an optional but non-limiting implementation, the scatter points in the difference scatter plot information are marked based on the first global image features and the second global image features of each target image, including but not limited to steps E1-E2:

[0112] Step E1: Calculate the feature similarity between each pair of target images based on the first global image features and the second global image features of each target image.

[0113] In this embodiment, pre-trained and post-trained global feature extraction models are used to extract first and second global image features from each target image. Then, based on the first and second global image features, the feature similarity between each pair of target images is calculated. Feature similarity can be used to characterize the similarity between two target images. For example, the distance between image features can be calculated using methods such as L2 distance or cosine distance to determine the feature similarity between each pair of target images. If cosine distance is used, the value range of feature similarity can be defined as 0-1, with values ​​closer to 1 indicating high feature similarity and values ​​closer to 0 indicating low feature similarity.

[0114] Step E2: Mark the scatter points corresponding to each pair of target objects in the difference scatter plot information based on the feature similarity between each pair of target images.

[0115] In this embodiment, after determining the feature similarity between pairs of target images, the scatter points corresponding to each pair of target objects in the difference scatter plot information can be further labeled based on the feature similarity. Specifically, scatter points with the same feature similarity are assigned the same label value, and scatter points with different feature similarities are assigned different label values.

[0116] Using the above method, the scatter points corresponding to each pair of target objects in the difference scatter plot information can be quickly and accurately marked based on the feature similarity between each pair of target images.

[0117] S350. Based on the label values ​​of each scatter point in the difference scatter plot information, perform quality inspection on the retrieval label information of the target image.

[0118] As an optional but non-limiting implementation, the retrieval tag information of the target image is quality checked based on the label values ​​of each scatter point in the difference scatter plot information, including but not limited to steps F1-F2:

[0119] Step F1: Determine the location distribution of each scatter point in the difference scatter plot information.

[0120] In this embodiment, the location distribution of each scatter point in the difference scatter plot information is first determined, that is, the location of each scatter point in the difference scatter plot information is determined.

[0121] Step F2: Determine the labeling quality of the target image's retrieval label information based on the location distribution of each scatter point. The closer the location distribution of the scatter points is to the origin, the higher the labeling quality of the target image's retrieval label information.

[0122] In this embodiment, after determining the positional distribution of each scatter point in the difference scatter plot information, the annotation quality of the retrieval label information of the target image can be further determined. The closer the scatter points are to the origin, the smaller the positional and angular differences between the target images; in this case, the more similar the target images are, and the higher the annotation quality of the retrieval label information of the target images.

[0123] Using the above method, based on the positional distribution of each scatter point in the difference scatter plot information, the quality of the retrieval label information of the target image can be quickly and accurately determined.

[0124] As an optional but non-limiting implementation, the annotation quality of the target image's retrieval label information is determined based on the positional distribution of each scatter point, including but not limited to steps G1-G2:

[0125] Step G1: Determine the percentage of scattered points distributed within the preset range of the origin based on the location distribution of each scattered point.

[0126] The preset range can refer to a pre-defined area near the origin. The percentage of scattered points can be the ratio of the number of scattered points within the preset range of the origin to the total number of scattered points in the difference scatter plot. It can be understood that a larger percentage indicates a more concentrated distribution of scattered points within the preset range of the origin.

[0127] Step G2: Based on the proportion of scattered points distributed within the preset range of the origin and the label value of the corresponding scattered points, determine the annotation quality of the retrieval label information of the target image. The larger the label value of the corresponding scattered points, the higher the annotation quality of the retrieval label information of the target image.

[0128] In this embodiment, if the scatter plot near the origin is concentrated with high feature similarity (large label value) and the scatter plot far from the origin has low feature similarity (small label value), it indicates that the labeling quality of the target image's retrieval label information is good; otherwise, it indicates that the labeling quality is poor.

[0129] By using the above method, based on the proportion of scattered points distributed within a preset range of the origin and the corresponding label values ​​of the scattered points, the annotation quality of the retrieval label information of the target image can be judged more accurately.

[0130] The technical solution of this disclosure generates difference scatter plot information based on the target positioning information of each target image. Each scatter point in the difference scatter plot information corresponds to a pair of target images, and each scatter point is determined by the positional and angular differences between the pair of target images. Based on the first and second global image features of each target image, the scatter points in the difference scatter plot information are marked, and the mark value of each scatter point is determined by the similarity of the global image features between the pair of target images corresponding to that scatter point. The retrieval tag information of the target image is quality checked based on the mark values ​​of each scatter point in the difference scatter plot information. By adopting the technical solution of this disclosure, the retrieval tag information of the target image is quickly and effectively quality checked by introducing reference quality check information to verify whether the annotation quality of the retrieval tag information meets the requirements of image retrieval. This helps reduce the cost of manual quality check, further accelerates the data iteration process in the image retrieval process, and improves the data iteration efficiency. Furthermore, by determining the difference scatter plot information through the positional and angular differences between every two target images, and by quality checking the retrieval tag information of the target image based on the mark values ​​of each scatter point in the difference scatter plot information, the accuracy of the retrieval tag information quality check is further improved.

[0131] Figure 4 This is a flowchart illustrating yet another image retrieval tag quality inspection method provided in this disclosure. This disclosure further optimizes the aforementioned embodiments, and can be combined with various optional solutions from one or more of the above embodiments. Figure 4 As shown, the image retrieval tag quality inspection method provided in this embodiment may include the following steps:

[0132] S410. Determine the retrieval tag quality inspection condition information for each target image in the target image sequence.

[0133] S420. Based on the retrieval tag quality inspection condition information, determine the reference quality inspection information required when inspecting the target retrieval tag information of the target image. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image.

[0134] S430. Based on the target retrieval label information, first global image features, and second global image features of each target image, determine the mean and variance of the positioning information of multiple categories of images. The mean and variance of the positioning information of each category of images include values ​​determined by different calculation methods. Refer to the data of each dimension in the quality inspection information to find a corresponding calculation method.

[0135] As an optional but non-limiting implementation, the mean and variance of the localization information of multiple categories of images are determined based on the target retrieval label information, the first global image features, and the second global image features of each target image, including but not limited to steps H1-H3:

[0136] Step H1: Based on the target location included in the target retrieval label information of each target image, calculate the first mean and first variance of the target location of the target image belonging to each category of image. The target location is used to describe the horizontal and vertical coordinates of the image acquisition location of the target image.

[0137] Here, the first mean and the first variance can refer to the mean and variance of the target position of each target image in each category of images determined based on the target retrieval tag information, respectively. In this embodiment, firstly, target images with the same target retrieval tag information are counted and classified into the same category. Then, for each category of images, the mean (i.e., the first mean) and variance (i.e., the first variance) of the target position of each target image in each category are calculated respectively.

[0138] Step H2: Perform first feature clustering analysis on each target image based on the first global image features of each target image, and calculate the second mean and second variance of the target positions of the target images in the same cluster.

[0139] Here, the second mean and the second variance can refer to the mean and variance of the target position of each target image in each clustered image determined based on the first global image features, respectively. In this embodiment, the global feature extraction model before training is first used to extract the first global image features for each target image. Then, clustering analysis is performed on each target image based on the first global image features. Target images clustered into the same category are then statistically analyzed, and the mean (i.e., the second mean) and variance (i.e., the second variance) of the target position of the target images in the same cluster are calculated.

[0140] Step H3: Perform second feature clustering analysis on each target image based on the second global image features of each target image, and calculate the third mean and third difference of the target positions of the target images in the same cluster.

[0141] The third mean and third variance refer to the mean and variance of the target position of each target image in each cluster, determined based on the second global image features. In this embodiment, the trained global feature extraction model is first used to extract the second global image features from each target image. Then, clustering analysis is performed on each target image based on the second global image features. Target images clustered into the same category are then statistically analyzed, and the mean (third mean) and variance (third variance) of the target position of the target images in the same cluster are calculated.

[0142] Using the above method, the mean and variance of the localization information of multiple categories of images can be quickly and accurately determined based on the target retrieval label information, the first global image features, and the second global image features of each target image.

[0143] S440. Based on the mean and variance of the location information of multiple categories of images determined by different calculation methods, perform quality inspection on the retrieval label information of the target image.

[0144] As an optional but non-limiting implementation, the retrieval label information of the target image is quality checked based on the mean and variance of the localization information of multiple categories of images determined by different calculation methods, including but not limited to steps I1-I4:

[0145] Step I1: Combine the mean and variance of multiple categories of images obtained by the same calculation method to obtain the mean set and variance set corresponding to each calculation method.

[0146] For example, combining the first mean and the first variance obtained in step H1 yields the set of means and variances for the calculation method corresponding to step H1. Combining the second mean and the second variance obtained in step H2 yields the set of means and variances for the calculation method corresponding to step H2. Combining the third mean and the third variance obtained in step H3 yields the set of means and variances for the calculation method corresponding to step H3.

[0147] Step I2: Perform a target matching operation on the mean sets corresponding to different calculation methods. The target matching operation is used to find a corresponding element in one mean set for each element in another mean set so that the total error of the target position mean between the corresponding elements of the two mean sets is minimized.

[0148] In this embodiment, after obtaining the mean set and variance set corresponding to each calculation method, a target matching operation can be performed on the mean sets corresponding to different calculation methods. The target matching operation is used to find a corresponding element in one mean set for each element in another mean set, such that the total error of the target position mean between corresponding elements in the two mean sets is minimized. See also... Figure 5 The left and right columns represent two sets of means, with each rectangle representing an element (i.e., a mean) within that set. Figure 5 In this case, although 7 and 8 are closer, since only 8 is closest to 10, 7 can only be matched with the second closest element, 5, to minimize the total error of the target position mean between corresponding elements of the two mean sets. For example, the Hungarian algorithm can be used to perform target matching on the mean sets.

[0149] Step I3: Determine the variance ratio and total error of the means that match successfully in the two mean sets.

[0150] In this embodiment, the variances of the means that successfully match in the two mean sets are first divided to determine the variance ratio. Then, the variance ratios of each pair of mean sets are summed to determine the variance ratio sum. Simultaneously, the differences between the two successfully matching means in the two mean sets are summed to determine the total error corresponding to the two mean sets.

[0151] Step I4: Based on the variance ratios corresponding to the means of successful mean matching and the total error, perform quality inspection on the retrieval label information of the target image.

[0152] In this embodiment, after determining the variance ratio and total error corresponding to the means of successful mean matching, the retrieval label information of the target image can be further quality checked. The closer the total error is to 0 and the closer the variance ratio is to 1, the better the labeling quality of the retrieval label information.

[0153] Using the above method, the retrieval label information of the target image can be quickly and accurately inspected based on the variance ratio and total error corresponding to the means that successfully match the means in two mean sets.

[0154] The technical solution of this disclosure determines the mean and variance of the location information of multiple categories of images based on the target retrieval tag information, the first global image features, and the second global image features of each target image. The mean and variance of the location information of each category of images includes values ​​determined by different calculation methods, with each dimension of the reference quality inspection information corresponding to a calculation method. The retrieval tag information of the target image is quality inspected based on the mean and variance of the location information of multiple categories of images determined by different calculation methods. By introducing reference quality inspection information to quickly and effectively inspect the retrieval tag information of the target image, verifying whether the labeling quality of the retrieval tag information meets the requirements of image retrieval, this technical solution helps reduce manual quality inspection costs, further accelerates the data iteration process in image retrieval, and improves data iteration efficiency. Furthermore, it can perform quality inspection on the retrieval tag information of the target image based on the mean and variance of the location information of multiple categories of images determined by different calculation methods, further improving the accuracy and efficiency of retrieval tag information quality inspection.

[0155] As an optional but non-limiting implementation, the retrieval tag information of the target image is quality checked based on the reference quality inspection information, and may include, but is not limited to, steps J1-J3:

[0156] Step J1: Perform first feature clustering analysis on each target image based on the first global image features of each target image, and calculate the average first similarity between each cluster center.

[0157] The first average similarity can refer to the average similarity between cluster centers after the first feature clustering analysis. In this embodiment, after performing the first feature clustering analysis on each target image, the first average similarity between each cluster center can be further calculated.

[0158] Step J2: Perform second feature clustering analysis on each target image based on the second global image features of each target image, and calculate the average second similarity between each cluster center.

[0159] The second average similarity can refer to the average similarity between cluster centers after the second feature clustering analysis. In this embodiment, after performing the second feature clustering analysis on each target image, the second average similarity between each cluster center can be further calculated.

[0160] Step J3: Perform quality inspection on the retrieval label information of the target image based on the average first similarity between each cluster center and the average second similarity between each cluster center.

[0161] In this embodiment, the smaller the average similarity between each cluster center, the lower the inter-class similarity, which can more accurately distinguish different categories. At this time, the labeling quality of the retrieved tag information is better.

[0162] By using the above method, the labeling quality of the retrieved tag information can be quickly and accurately determined based on the average similarity between each cluster center, further improving the accuracy of the quality inspection of the retrieved tag information.

[0163] Figure 6 This is a flowchart illustrating yet another image retrieval tag quality inspection method provided in this disclosure. This disclosure further optimizes the aforementioned embodiments, and can be combined with various optional solutions from one or more of the above embodiments. Figure 6 As shown, the image retrieval tag quality inspection method provided in this embodiment may include the following steps:

[0164] S510. Determine the retrieval tag quality inspection condition information for each target image in the target image sequence.

[0165] S520. Based on the retrieval tag quality inspection condition information, determine the reference quality inspection information required when performing quality inspection on the target retrieval tag information of the target image. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image.

[0166] S530. Based on the target category label included in the target retrieval label information of each target image, determine the retrieval target image that is closest to the cluster center from the target images of each category label.

[0167] The target image used for retrieval can refer to the target image used as a reference for retrieval. In this embodiment, for each category of the target category tags included in the target retrieval tag information, the target image closest to the cluster center is selected as the target image used for retrieval.

[0168] S540. Determine the second global image features of the target image for retrieval, and combine the second global image features of the target image for retrieval with the target positioning information of the target image for retrieval to form retrieval data.

[0169] In this embodiment, the trained global feature extraction model is first used to extract the second global image features of the target image for retrieval. Then, the second global image features of the target image for retrieval and the target positioning information (including target position and target angle) of the target image for retrieval are used to form retrieval data.

[0170] S550. Use the remaining target images of each category label as query target images, and determine the second global image features of the query target images.

[0171] The target image for querying can refer to the target image used for querying. In this embodiment, the remaining target images of each category label are used as the target images for querying, and then the trained global feature extraction model is used to extract the second global image features of the target images for querying.

[0172] S560. Calculate the similarity between the target image used for querying and all target images used for retrieval in the retrieval data, in order to perform quality inspection on the retrieval tag information of the target image.

[0173] In this embodiment, the similarity between the query target image and all retrieval target images in the retrieval data is calculated, and the similarity scores are sorted in descending order. The retrieval tag information of the target images can then be quality checked based on the similarity ranking results. For example, the retrieval tag information of the target images can be quality checked based on the average accuracy rate of the ranked similarity results. Specifically, the retrieval target images corresponding to the top k similarity scores in the similarity ranking results are used as the retrieval results for the current query target image. Then, the ratio of the number of target images in the retrieval results that have the same category tag as the current query target image to k (i.e., the average accuracy rate) is calculated. The closer the average accuracy rate is to 1, the higher the retrieval accuracy, indicating better labeling quality of the retrieval tag information. The value of k is not greater than the total number of all retrieval target images in the retrieval data. This embodiment does not limit the specific value of k; it can be set according to actual needs, for example, k can be 1, 3, 5, etc.

[0174] For example, the quality control of the retrieval label information of the target image can be performed based on the average recall rate according to the similarity ranking results. Specifically, the retrieval target images corresponding to the top k similarity scores in the similarity ranking results are used as the retrieval results for the current query target image. If there is a retrieval target image with the same category label as the current query target image in the retrieval results, it is considered that the current query target image has a correct query in the retrieval results, and is recorded as a correct retrieval. Then, the ratio of the total number of correctly retrieved query target images to the total number of query target images (i.e., the average recall rate) is calculated. The closer the average recall rate is to 1, the higher the retrieval recall rate, indicating that the labeling quality of the retrieval label information is better.

[0175] The technical solution of this disclosure, based on the target category labels included in the target retrieval label information of each target image, determines a retrieval target image closest to the cluster center from the target images of each category label; determines the second global image feature of the retrieval target image, and constructs retrieval data by combining the second global image feature of the retrieval target image with the target positioning information of the retrieval target image; uses the remaining target images of each category label as query target images, and determines the second global image feature of the query target images; calculates the similarity between the query target image and all retrieval target images in the retrieval data, and uses this similarity to perform quality inspection on the retrieval label information of the target images. By adopting the technical solution of this disclosure, the retrieval label information of the target images is quickly and effectively inspected by introducing reference quality inspection information to verify whether the labeling quality of the retrieval label information meets the requirements of image retrieval. This helps reduce the cost of manual quality inspection, further accelerates the data iteration process in the image retrieval process, and improves the data iteration efficiency. Furthermore, based on the similarity between the query target image and all retrieval target images in the retrieval data, the retrieval label information of the target images can be inspected, further improving the efficiency and accuracy of quality inspection.

[0176] Figure 7 This is a schematic diagram of the structure of an image retrieval tag quality inspection device provided in an embodiment of this disclosure. This embodiment is applicable to situations requiring rapid and effective quality inspection of image retrieval tags. The device can be implemented in software and / or hardware and is generally integrated into any electronic device with network communication capabilities, including but not limited to mobile terminals, PCs, or servers. Figure 7 As shown, the device includes: a label quality inspection condition information determination module 610, a reference quality inspection information determination module 620, and a label information quality inspection module 630; wherein:

[0177] The retrieval tag quality inspection condition information determination module 610 is used to determine the retrieval tag quality inspection condition information of each target image in the target image sequence;

[0178] The reference quality inspection information determination module 620 is used to determine the reference quality inspection information required when performing quality inspection on the target retrieval tag information of the target image based on the retrieval tag quality inspection condition information. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image.

[0179] The retrieval tag information quality inspection module 630 is used to perform quality inspection on the retrieval tag information of the target image based on the reference quality inspection information.

[0180] In one optional embodiment of this disclosure, the retrieval tag quality inspection condition information may include retrieval tag usage scenarios and retrieval tag quality inspection methods. The retrieval tag usage scenarios may include using the global image features of the target image for coarse location on a known map and using the global image features of the target image to determine whether a preset scenario has been reached. The retrieval tag quality inspection methods may be used to indicate the quality inspection process for the target retrieval tag information of the target image under the retrieval tag usage scenarios.

[0181] In one optional embodiment of this disclosure, the retrieval tag information quality inspection module 630 may include:

[0182] The target scatter plot information determination unit is used to generate target scatter plot information based on the target location included in the target positioning information of each target image. The target location is used to describe the horizontal and vertical coordinates of the image acquisition location of the target image. Each scatter point in the target scatter plot information corresponds to one target image.

[0183] The first scatter point marking unit is used to mark the scatter points in the target scatter plot information based on the target retrieval label information of each target image, the first global image feature and the second global image feature. The same scatter point in the target scatter plot information is marked with different marking operations. Each dimension of the data in the reference quality inspection information corresponds to a marking operation.

[0184] The first retrieval tag information quality inspection unit is used to perform quality inspection on the retrieval tag information of the target image based on the tag values ​​obtained by applying different tagging operations to each scatter point in the target scatter plot information.

[0185] In one optional embodiment of this disclosure, the first scatter marker unit may include:

[0186] The first labeling operation execution subunit is used to perform a first labeling operation on the scatter points corresponding to the target image in the target scatter plot information based on the target category label included in the target retrieval label information of the target image, wherein the label value of the first labeling operation matches the target category label of the target image;

[0187] The second labeling operation execution subunit is used to determine the first reference category label of each target image based on the first global image features of each target image, and to perform a second labeling operation on the scatter points corresponding to the target images in the target scatter plot information based on the first reference category label of the target images, wherein the label value of the second labeling operation matches the first reference category label of the target images;

[0188] The third labeling operation execution subunit is used to determine the second reference category label of each target image based on the second global image features of each target image, and to perform a third labeling operation on the scatter points corresponding to the target images in the target scatter plot information based on the second reference category label of the target images, wherein the label value of the third labeling operation matches the second reference category label of the target images.

[0189] In one optional embodiment of this disclosure, the second marking operation execution subunit is optionally configured to:

[0190] Based on the first global image features of each target image, a first feature clustering analysis is performed on each target image, and target images in the same cluster are assigned the same first reference category label;

[0191] The third marking operation execution subunit is used for:

[0192] Based on the second global image features of each target image, a second feature clustering analysis is performed on each target image, and target images in the same cluster are assigned the same second reference category label.

[0193] In one optional embodiment of this disclosure, the first retrieval tag information quality inspection unit is optionally configured to:

[0194] When any marking operation is used for scatter point marking, the positional distribution of scatter points with the same mark value in the target scatter plot information is determined.

[0195] If all scattered points with the same label value are concentrated in a preset size and location area, then the labeling quality of the retrieval label information of the target image is determined to meet the preset quality conditions.

[0196] In one optional embodiment of this disclosure, the first retrieval tag information quality inspection unit may further be used to:

[0197] When the first marking operation is used to mark scatter points, the first position distribution of each scatter point with different marking values ​​in the target scatter plot information is determined.

[0198] When it is determined that the second marking operation is used for scatter point marking, the second position distribution of each scatter point with different marking values ​​in the target scatter plot information;

[0199] When it is determined that the third marking operation is used for scatter point marking, the third position distribution of each scatter point with different marking values ​​in the target scatter plot information;

[0200] The labeling quality of the retrieval tag information of the target image is determined based on the overlap between the first and second position distributions of the same label value and the overlap between the first and third position distributions of the same label value. The higher the overlap between the two position distributions, the higher the labeling quality of the retrieval tag information of the target image.

[0201] In one optional embodiment of this disclosure, the retrieval tag information quality inspection module 630 may further include:

[0202] The difference scatter plot information generation unit is used to generate difference scatter plot information based on the target positioning information of each target image. Each scatter point in the difference scatter plot information corresponds to a pair of target images. Each scatter point in the difference scatter plot information is determined by the positional and angular differences between a pair of target images.

[0203] The second scatter point labeling unit is used to label the scatter points in the difference scatter plot information according to the first global image features and the second global image features of each target image. The label value of the scatter point is determined by the similarity of global image features between the pair of target images corresponding to the scatter point.

[0204] The second retrieval tag information quality inspection unit is used to perform quality inspection on the retrieval tag information of the target image based on the label values ​​of each scatter point in the difference scatter plot information.

[0205] In one optional embodiment of this disclosure, the difference scatter plot information generation unit is optionally configured to:

[0206] Based on the target position and target angle included in the target positioning information of each target image, determine the absolute value of the position difference and the absolute value of the angle difference between any two target images;

[0207] A difference scatter plot is constructed using the absolute values ​​of the positional differences and the absolute values ​​of the angle differences between each pair of target images as the horizontal and vertical coordinates. The target position is used to describe the horizontal and vertical coordinates of the image acquisition position of the target image, and the target angle describes the acquisition direction when the target image is acquired.

[0208] In one optional embodiment of this disclosure, the second scatter marker unit is optionally used for:

[0209] Based on the first global image features and the second global image features of each target image, the feature similarity between each pair of target images is calculated;

[0210] The scatter points corresponding to each pair of target objects in the difference scatter plot information are marked based on the feature similarity between each pair of target images.

[0211] In one optional embodiment of this disclosure, the second retrieval tag information quality inspection unit may include:

[0212] The scatter point location distribution determination subunit is used to determine the location distribution of each scatter point in the difference scatter plot information;

[0213] The annotation quality determination subunit is used to determine the annotation quality of the retrieval tag information of the target image based on the positional distribution of each scatter point. The closer the positional distribution of the scatter points is to the origin, the higher the annotation quality of the retrieval tag information of the target image.

[0214] In one optional embodiment of this disclosure, the annotation quality determination subunit is optionally configured to:

[0215] The proportion of scattered points distributed within a preset range of the origin is determined based on the location distribution of each scattered point;

[0216] The annotation quality of the retrieval tag information of the target image is determined based on the proportion of scattered points distributed within a preset range of the origin and the label value of the corresponding scattered points. The larger the label value of the corresponding scattered points, the higher the annotation quality of the retrieval tag information of the target image.

[0217] In one optional embodiment of this disclosure, the retrieval tag information quality inspection module 630 may further include:

[0218] The mean and variance determination unit is used to determine the mean and variance of the positioning information of multiple categories of images based on the target retrieval label information of each target image, the first global image feature and the second global image feature. The mean and variance of the positioning information of each category of images include values ​​determined by different calculation methods. Each dimension of the data in the reference quality inspection information corresponds to a calculation method.

[0219] The third retrieval tag information quality inspection unit is used to perform quality inspection on the retrieval tag information of the target image based on the mean and variance of the positioning information of multiple categories of images determined by different calculation methods.

[0220] In one optional embodiment of this disclosure, the mean and variance determination unit is optionally configured to:

[0221] Based on the target location included in the target retrieval tag information of each target image, calculate the first mean and first variance of the target location of the target image belonging to each category of image, wherein the target location is used to describe the horizontal and vertical coordinates of the image acquisition location of the target image;

[0222] Based on the first global image features of each target image, a first feature clustering analysis is performed on each target image, and the second mean and second variance of the target positions of the target images in the same cluster are calculated;

[0223] Based on the second global image features of each target image, a second feature clustering analysis is performed on each target image, and the third mean and third difference of the target positions of the target images in the same cluster are calculated.

[0224] In one optional embodiment of this disclosure, the third retrieval tag information quality inspection unit is optionally configured to:

[0225] The mean and variance of multiple categories of images obtained by the same calculation method are combined to obtain the mean set and variance set corresponding to each calculation method.

[0226] The target matching operation is performed on the mean sets corresponding to different calculation methods. The target matching operation is used to find a corresponding element in one mean set for each element in another mean set so that the total error of the target position mean between the corresponding elements of the two mean sets is minimized.

[0227] Determine the ratio of variances and the total error of the means that match successfully in two sets of means;

[0228] The retrieval tag information of the target image is quality inspected based on the variance ratio corresponding to the mean of the successful mean matching and the total error.

[0229] In one optional embodiment of this disclosure, the tag information retrieval quality inspection module 630 may further be used for:

[0230] Based on the first global image features of each target image, a first feature clustering analysis is performed on each target image, and the average first similarity between each cluster center is calculated;

[0231] Based on the second global image features of each target image, a second feature clustering analysis is performed on each target image, and the average second similarity between each cluster center is calculated;

[0232] The retrieval tag information of the target image is quality inspected based on the average first similarity among the cluster centers and the average second similarity among the cluster centers.

[0233] In one optional embodiment of this disclosure, the tag information retrieval quality inspection module 630 may further be used for:

[0234] Based on the target category labels included in the target retrieval label information of each target image, a retrieval target image closest to the cluster center is determined from the target images of each category label;

[0235] Determine the second global image feature of the target image for retrieval, and combine the second global image feature of the target image for retrieval with the target positioning information of the target image for retrieval to form retrieval data;

[0236] The remaining target images for each category label are used as query target images, and the second global image features of the query target images are determined.

[0237] The similarity between the target image used for querying and all target images used for retrieval in the retrieval data is calculated to perform quality checks on the retrieval tag information of the target image.

[0238] The image retrieval tag quality inspection device provided in this embodiment can execute the image retrieval tag quality inspection method provided in this embodiment, and has the corresponding functional modules and beneficial effects of executing the method.

[0239] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0240] Figure 8 This is a schematic diagram of the structure of an image retrieval tag quality inspection electronic device provided in an embodiment of this disclosure. Refer to the following... Figure 8 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 8The diagram below shows the structure of the terminal device or server 500. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0241] like Figure 8 As shown, electronic device 500 may include a processing unit (e.g., central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An edit / output (I / O) interface 505 is also connected to bus 504.

[0242] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0243] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0244] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0245] The electronic device provided in this embodiment and the image retrieval tag quality inspection method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0246] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the image retrieval tag quality inspection method provided in the above embodiments.

[0247] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0248] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0249] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0250] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0251] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: respond to a touch operation request, determine target touch operation information associated with the target device; determine, based on the target touch operation information, customary interaction triggering condition information adapted when the target device performs a target interactive operation; and perform adaptive control on the target interactive operation of the target device based on the customary interaction triggering condition information.

[0252] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0253] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0254] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0255] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0256] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0257] According to one or more embodiments of this disclosure, Example 1 provides an image retrieval tag quality inspection method, the method comprising:

[0258] Determine the retrieval tag quality inspection condition information for each target image in the target image sequence;

[0259] Based on the retrieval tag quality inspection condition information, the reference quality inspection information required for quality inspection of the target retrieval tag information of the target image is determined. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image, respectively.

[0260] The retrieval tag information of the target image is inspected based on the reference quality inspection information.

[0261] Example 2, based on the method described in Example 1, the retrieval tag quality inspection condition information includes retrieval tag usage scenarios and retrieval tag quality inspection methods. The retrieval tag usage scenarios include using the global image features of the target image for coarse positioning of the geographical location on a known map and using the global image features of the target image to determine whether a preset scenario has been reached. The retrieval tag quality inspection methods are used to indicate the quality inspection process for the target retrieval tag information of the target image under the retrieval tag usage scenarios.

[0262] Example 3, based on the method described in Example 2, performs quality inspection on the retrieval tag information of the target image according to the reference quality inspection information, including:

[0263] Target scatter plot information is generated based on the target location included in the target positioning information of each target image. The target location is used to describe the horizontal and vertical coordinates of the image acquisition location of the target image. Each scatter point in the target scatter plot information corresponds to one target image.

[0264] Based on the target retrieval label information of each target image, the first global image feature, and the second global image feature, the scattered points in the target scatter plot information are marked. The same scattered point in the target scatter plot information is marked using different marking operations. Each dimension of the data in the reference quality inspection information corresponds to a marking operation.

[0265] Based on the label values ​​obtained by applying different labeling operations to each scatter point in the target scatter plot information, the retrieval label information of the target image is subjected to quality inspection.

[0266] Example 4, according to the method described in Example 3, marks the scatter points in the target scatter plot information based on the target retrieval label information of each target image, the first global image feature, and the second global image feature, including:

[0267] Based on the target category label included in the target retrieval label information of the target image, a first labeling operation is performed on the scatter points corresponding to the target image in the target scatter plot information, and the label value of the first labeling operation matches the target category label of the target image;

[0268] A first reference category label for each target image is determined based on the first global image features of each target image, and a second labeling operation is performed on the scatter points corresponding to the target images in the target scatter plot information based on the first reference category label of the target images, wherein the label value of the second labeling operation matches the first reference category label of the target images;

[0269] A second reference category label is determined for each target image based on the second global image features of each target image, and a third labeling operation is performed on the scatter points corresponding to the target images in the target scatter plot information based on the second reference category label of the target images, wherein the label value of the third labeling operation matches the second reference category label of the target images.

[0270] Example 5, according to the method described in Example 4, determines a first reference category label for each of the target images based on the first global image features of each target image, including:

[0271] Based on the first global image features of each target image, a first feature clustering analysis is performed on each target image, and target images in the same cluster are assigned the same first reference category label;

[0272] Determining a second reference category label for each target image based on the second global image features of each target image includes:

[0273] Based on the second global image features of each target image, a second feature clustering analysis is performed on each target image, and target images in the same cluster are assigned the same second reference category label.

[0274] Example 6, based on the method described in Example 3, performs quality inspection on the retrieval tag information of the target image according to the tag values ​​obtained by applying different tagging operations to each scatter point in the target scatter plot information, including:

[0275] When any marking operation is used for scatter point marking, the positional distribution of scatter points with the same mark value in the target scatter plot information is determined.

[0276] If all scattered points with the same label value are concentrated in a preset size and location area, then the labeling quality of the retrieval label information of the target image is determined to meet the preset quality conditions.

[0277] Example 7, based on the method described in Example 4, performs quality inspection on the retrieval tag information of the target image according to the tag values ​​obtained by applying different tagging operations to each scatter point in the target scatter plot information, including:

[0278] When the first marking operation is used to mark scatter points, the first position distribution of each scatter point with different marking values ​​in the target scatter plot information is determined.

[0279] When it is determined that the second marking operation is used for scatter point marking, the second position distribution of each scatter point with different marking values ​​in the target scatter plot information;

[0280] When it is determined that the third marking operation is used for scatter point marking, the third position distribution of each scatter point with different marking values ​​in the target scatter plot information;

[0281] The labeling quality of the retrieval tag information of the target image is determined based on the overlap between the first and second position distributions of the same label value and the overlap between the first and third position distributions of the same label value. The higher the overlap between the two position distributions, the higher the labeling quality of the retrieval tag information of the target image.

[0282] Example 8, according to the method described in Example 2, performs quality inspection on the retrieval tag information of the target image based on the reference quality inspection information, including:

[0283] Based on the target positioning information of each target image, a difference scatter plot is generated. Each scatter point in the difference scatter plot corresponds to a pair of target images. Each scatter point in the difference scatter plot is determined by the positional and angular differences between a pair of target images.

[0284] Based on the first global image features and the second global image features of each target image, the scatter points in the difference scatter plot information are marked, and the marking value of the scatter points is determined by the similarity of global image features between the pair of target images corresponding to the scatter points;

[0285] Based on the label values ​​of each scatter point in the difference scatter plot information, the retrieval tag information of the target image is quality inspected.

[0286] Example 9, according to the method described in Example 8, generates difference scatter plot information based on the target localization information of each target image, including:

[0287] Based on the target position and target angle included in the target positioning information of each target image, determine the absolute value of the position difference and the absolute value of the angle difference between any two target images;

[0288] A difference scatter plot is constructed using the absolute values ​​of the positional differences and the absolute values ​​of the angle differences between each pair of target images as the horizontal and vertical coordinates. The target position is used to describe the horizontal and vertical coordinates of the image acquisition position of the target image, and the target angle describes the acquisition direction when the target image is acquired.

[0289] Example 10, according to the method described in Example 8, marks the scatter points in the difference scatter plot information based on the first global image features and the second global image features of each of the target images, including:

[0290] Based on the first global image features and the second global image features of each target image, the feature similarity between each pair of target images is calculated;

[0291] The scatter points corresponding to each pair of target objects in the difference scatter plot information are marked based on the feature similarity between each pair of target images.

[0292] Example 11, according to the method described in Example 8, performs quality inspection on the retrieval tag information of the target image based on the label values ​​of each scatter point in the difference scatter plot information, including:

[0293] Determine the positional distribution of each scatter point in the difference scatter plot information;

[0294] The labeling quality of the retrieval label information of the target image is determined based on the positional distribution of each scatter point. The closer the positional distribution of the scatter points is to the origin, the higher the labeling quality of the retrieval label information of the target image.

[0295] Example 12, based on the method described in Example 11, determines the annotation quality of the retrieval label information of the target image according to the positional distribution of each scatter point, including:

[0296] The proportion of scattered points distributed within a preset range of the origin is determined based on the location distribution of each scattered point;

[0297] The annotation quality of the retrieval tag information of the target image is determined based on the proportion of scattered points distributed within a preset range of the origin and the label value of the corresponding scattered points. The larger the label value of the corresponding scattered points, the higher the annotation quality of the retrieval tag information of the target image.

[0298] Example 13, according to the method described in Example 2, performs quality inspection on the retrieval tag information of the target image based on the reference quality inspection information, including:

[0299] Based on the target retrieval tag information of each target image, the first global image feature, and the second global image feature, the mean and variance of the positioning information of multiple categories of images are determined. The mean and variance of the positioning information of each category of images include values ​​determined by different calculation methods. Each dimension of the data in the reference quality inspection information corresponds to a calculation method.

[0300] The retrieval tag information of the target image is quality checked based on the mean and variance of the location information of multiple categories of images determined by different calculation methods.

[0301] Example 14, according to the method described in Example 13, determines the mean and variance of the localization information of multiple categories of images based on the target retrieval label information of each target image, the first global image feature, and the second global image feature, including:

[0302] Based on the target location included in the target retrieval tag information of each target image, calculate the first mean and first variance of the target location of the target image belonging to each category of image, wherein the target location is used to describe the horizontal and vertical coordinates of the image acquisition location of the target image;

[0303] Based on the first global image features of each target image, a first feature clustering analysis is performed on each target image, and the second mean and second variance of the target positions of the target images in the same cluster are calculated;

[0304] Based on the second global image features of each target image, a second feature clustering analysis is performed on each target image, and the third mean and third difference of the target positions of the target images in the same cluster are calculated.

[0305] Example 15, based on the method described in Example 13, performs quality inspection on the retrieval tag information of the target image according to the mean and variance of the location information of multiple categories of images determined by different calculation methods, including:

[0306] The mean and variance of multiple categories of images obtained by the same calculation method are combined to obtain the mean set and variance set corresponding to each calculation method.

[0307] The target matching operation is performed on the mean sets corresponding to different calculation methods. The target matching operation is used to find a corresponding element in one mean set for each element in another mean set so that the total error of the target position mean between the corresponding elements of the two mean sets is minimized.

[0308] Determine the ratio of variances and the total error of the means that match successfully in two sets of means;

[0309] The retrieval tag information of the target image is quality inspected based on the variance ratio corresponding to the mean of the successful mean matching and the total error.

[0310] Example 16, according to the method described in Example 13, further includes performing quality inspection on the retrieval tag information of the target image based on the reference quality inspection information:

[0311] Based on the first global image features of each target image, a first feature clustering analysis is performed on each target image, and the average first similarity between each cluster center is calculated;

[0312] Based on the second global image features of each target image, a second feature clustering analysis is performed on each target image, and the average second similarity between each cluster center is calculated;

[0313] The retrieval tag information of the target image is quality inspected based on the average first similarity among the cluster centers and the average second similarity among the cluster centers.

[0314] Example 17, according to the method described in Example 2, performs quality inspection on the retrieval tag information of the target image based on the reference quality inspection information, including:

[0315] Based on the target category labels included in the target retrieval label information of each target image, a retrieval target image closest to the cluster center is determined from the target images of each category label;

[0316] Determine the second global image feature of the target image for retrieval, and combine the second global image feature of the target image for retrieval with the target positioning information of the target image for retrieval to form retrieval data;

[0317] The remaining target images for each category label are used as query target images, and the second global image features of the query target images are determined.

[0318] The similarity between the target image used for querying and all target images used for retrieval in the retrieval data is calculated to perform quality checks on the retrieval tag information of the target image.

[0319] According to one or more embodiments of this disclosure, Example 18 also provides an image retrieval tag quality inspection device, the image retrieval tag quality inspection device comprising:

[0320] The retrieval tag quality inspection condition information determination module is used to determine the retrieval tag quality inspection condition information for each target image in the target image sequence;

[0321] The reference quality inspection information determination module is used to determine the reference quality inspection information required when inspecting the target retrieval tag information of the target image based on the retrieval tag quality inspection condition information. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image, respectively.

[0322] The tag information quality inspection module is used to perform quality inspection on the tag information of the target image based on the reference quality inspection information.

[0323] According to one or more embodiments of this disclosure, Example 19 also provides an image retrieval tag quality inspection electronic device, the electronic device comprising:

[0324] One or more processors;

[0325] Storage device for storing one or more programs.

[0326] When the one or more programs are executed by the one or more processors, the one or more processors implement the image retrieval tag quality inspection method as described in any of Examples 1-17.

[0327] According to one or more embodiments of this disclosure, Example 20 also provides a storage medium containing computer-executable instructions that, when executed by a computer processor, are used to perform an image retrieval tag quality inspection method as described in any of Examples 1-17.

[0328] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0329] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0330] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image retrieval tag quality inspection method, characterized in that, The method includes: Determine the retrieval tag quality inspection condition information for each target image in the target image sequence; the retrieval tag quality inspection condition information includes the retrieval tag usage scenario and the retrieval tag quality inspection method. The retrieval tag usage scenario includes using the global image features of the target image for coarse positioning on a known map and using the global image features of the target image to determine whether a preset scenario has been reached. The retrieval tag quality inspection method is used to indicate the quality inspection process for the target retrieval tag information of the target image under the retrieval tag usage scenario. Based on the retrieval tag quality inspection condition information, the reference quality inspection information required for quality inspection of the target retrieval tag information of the target image is determined. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image, respectively. The retrieval tag information of the target image is inspected based on the reference quality inspection information; The retrieval tag information of the target image is quality checked using at least one of the following methods: The quality inspection method based on target scatter plots includes: generating target scatter plot information based on the target location included in the target positioning information of each target image, wherein the target location is used to describe the horizontal and vertical coordinates of the image acquisition location of the target image, and each scatter point in the target scatter plot information corresponds to one target image; marking the scatter points in the target scatter plot information based on the target retrieval tag information, the first global image feature, and the second global image feature of each target image, wherein the same scatter point in the target scatter plot information is marked using different marking operations, and each dimension of the reference quality inspection information corresponds to a marking operation; and performing quality inspection on the retrieval tag information of the target image based on the marking values ​​obtained by applying different marking operations to each scatter point in the target scatter plot information. The quality inspection method based on the difference scatter plot includes: generating difference scatter plot information based on the target positioning information of each target image, wherein each scatter point in the difference scatter plot information corresponds to a pair of target images, and each scatter point in the difference scatter plot information is determined by the positional and angular differences between the pair of target images; marking the scatter points in the difference scatter plot information based on the first global image features and the second global image features of each target image, wherein the marking value of the scatter point is determined by the similarity of the global image features between the pair of target images corresponding to the scatter point; and performing quality inspection on the retrieval tag information of the target images based on the marking values ​​of each scatter point in the difference scatter plot information. The quality inspection method based on mean and variance includes: determining the mean and variance of the location information of multiple categories of images based on the target retrieval tag information of each target image, the first global image feature, and the second global image feature. The mean and variance of the location information of each category of images includes values ​​determined by different calculation methods, and each dimension of the data in the reference quality inspection information corresponds to a calculation method; and performing quality inspection on the retrieval tag information of the target image based on the mean and variance of the location information of multiple categories of images determined by different calculation methods. The similarity-based quality inspection method includes: determining a retrieval target image closest to the cluster center from the target images of each category label based on the target category label information included in the target retrieval label information of each target image; determining the second global image feature of the retrieval target image; constructing retrieval data by combining the second global image feature of the retrieval target image with the target positioning information of the retrieval target image; using the remaining target images of each category label as query target images, and determining the second global image feature of the query target images; calculating the similarity between the query target images and all retrieval target images in the retrieval data, in order to perform quality inspection on the retrieval label information of the target images.

2. The method according to claim 1, characterized in that, The scatter points in the target scatter plot information are marked based on the target retrieval label information of each target image, the first global image feature, and the second global image feature, including: Based on the target category label included in the target retrieval label information of the target image, a first labeling operation is performed on the scatter points corresponding to the target image in the target scatter plot information, and the label value of the first labeling operation matches the target category label of the target image; A first reference category label for each target image is determined based on the first global image features of each target image, and a second labeling operation is performed on the scatter points corresponding to the target images in the target scatter plot information based on the first reference category label of the target images, wherein the label value of the second labeling operation matches the first reference category label of the target images; A second reference category label is determined for each target image based on the second global image features of each target image, and a third labeling operation is performed on the scatter points corresponding to the target images in the target scatter plot information based on the second reference category label of the target images, wherein the label value of the third labeling operation matches the second reference category label of the target images.

3. The method according to claim 2, characterized in that, Determining a first reference category label for each target image based on the first global image features of each target image includes: Based on the first global image features of each target image, a first feature clustering analysis is performed on each target image, and target images in the same cluster are assigned the same first reference category label; Determining a second reference category label for each target image based on the second global image features of each target image includes: Based on the second global image features of each target image, a second feature clustering analysis is performed on each target image, and target images in the same cluster are assigned the same second reference category label.

4. The method according to claim 1, characterized in that, Based on the label values ​​obtained by applying different labeling operations to each scatter point in the target scatter plot information, the retrieval label information of the target image is quality checked, including: When any marking operation is used for scatter point marking, the positional distribution of scatter points with the same mark value in the target scatter plot information is determined. If all scattered points with the same label value are concentrated in a preset size and location area, then the labeling quality of the retrieval label information of the target image is determined to meet the preset quality conditions.

5. The method according to claim 2, characterized in that, Based on the label values ​​obtained by applying different labeling operations to each scatter point in the target scatter plot information, the retrieval label information of the target image is quality checked, including: When the first marking operation is used to mark scatter points, the first position distribution of each scatter point with different marking values ​​in the target scatter plot information is determined. When it is determined that the second marking operation is used for scatter point marking, the second position distribution of each scatter point with different marking values ​​in the target scatter plot information; When it is determined that the third marking operation is used for scatter point marking, the third position distribution of each scatter point with different marking values ​​in the target scatter plot information; The labeling quality of the retrieval tag information of the target image is determined based on the overlap between the first and second position distributions of the same label value and the overlap between the first and third position distributions of the same label value. The higher the overlap between the two position distributions, the higher the labeling quality of the retrieval tag information of the target image.

6. The method according to claim 1, characterized in that, Generate difference scatter plot information based on the target localization information of each target image, including: Based on the target position and target angle included in the target positioning information of each target image, determine the absolute value of the position difference and the absolute value of the angle difference between any two target images; A difference scatter plot is constructed using the absolute values ​​of the positional differences and the absolute values ​​of the angle differences between each pair of target images as the horizontal and vertical coordinates. The target position is used to describe the horizontal and vertical coordinates of the image acquisition position of the target image, and the target angle describes the acquisition direction when the target image is acquired.

7. The method according to claim 1, characterized in that, Based on the first global image features and the second global image features of each of the target images, the scatter points in the difference scatter plot information are marked, including: Based on the first global image features and the second global image features of each target image, the feature similarity between each pair of target images is calculated; The scatter points corresponding to each pair of target objects in the difference scatter plot information are marked based on the feature similarity between each pair of target images.

8. The method according to claim 1, characterized in that, Based on the label values ​​of each scatter point in the difference scatter plot information, the retrieval tag information of the target image is quality checked, including: Determine the positional distribution of each scatter point in the difference scatter plot information; The labeling quality of the retrieval label information of the target image is determined based on the positional distribution of each scatter point. The closer the positional distribution of the scatter points is to the origin, the higher the labeling quality of the retrieval label information of the target image.

9. The method according to claim 8, characterized in that, Determining the annotation quality of the retrieval tag information of the target image based on the positional distribution of each scatter point includes: The proportion of scattered points distributed within a preset range of the origin is determined based on the location distribution of each scattered point; The annotation quality of the retrieval tag information of the target image is determined based on the proportion of scattered points distributed within a preset range of the origin and the label value of the corresponding scattered points. The larger the label value of the corresponding scattered points, the higher the annotation quality of the retrieval tag information of the target image.

10. The method according to claim 1, characterized in that, The mean and variance of the localization information of multiple categories of images are determined based on the target retrieval label information of each target image, the first global image feature, and the second global image feature, including: Based on the target location included in the target retrieval tag information of each target image, calculate the first mean and first variance of the target location of the target image belonging to each category of image, wherein the target location is used to describe the horizontal and vertical coordinates of the image acquisition location of the target image; Based on the first global image features of each target image, a first feature clustering analysis is performed on each target image, and the second mean and second variance of the target positions of the target images in the same cluster are calculated; Based on the second global image features of each target image, a second feature clustering analysis is performed on each target image, and the third mean and third difference of the target positions of the target images in the same cluster are calculated.

11. The method according to claim 1, characterized in that, Based on the mean and variance of the location information of multiple categories of images determined by different calculation methods, the retrieval tag information of the target image is quality checked, including: The mean and variance of multiple categories of images obtained by the same calculation method are combined to obtain the mean set and variance set corresponding to each calculation method. The target matching operation is performed on the mean sets corresponding to different calculation methods. The target matching operation is used to find a corresponding element in one mean set for each element in another mean set so that the total error of the target position mean between the corresponding elements of the two mean sets is minimized. Determine the ratio of variances and the total error of the means that match successfully in two sets of means; The retrieval tag information of the target image is quality inspected based on the variance ratio corresponding to the mean of the successful mean matching and the total error.

12. The method according to claim 1, characterized in that, The quality inspection of the retrieval tag information of the target image based on the reference quality inspection information also includes: Based on the first global image features of each target image, a first feature clustering analysis is performed on each target image, and the average first similarity between each cluster center is calculated; Based on the second global image features of each target image, a second feature clustering analysis is performed on each target image, and the average second similarity between each cluster center is calculated; The retrieval tag information of the target image is quality inspected based on the average first similarity among the cluster centers and the average second similarity among the cluster centers.

13. An image retrieval tag quality inspection device, characterized in that, The device includes: The retrieval tag quality inspection condition information determination module is used to determine the retrieval tag quality inspection condition information for each target image in the target image sequence. The retrieval tag quality inspection condition information includes the retrieval tag usage scenario and the retrieval tag quality inspection method. The retrieval tag usage scenario includes using the global image features of the target image for coarse positioning on a known map and using the global image features of the target image to determine whether a preset scenario has been reached. The retrieval tag quality inspection method is used to indicate the quality inspection process for the target retrieval tag information of the target image under the retrieval tag usage scenario. The reference quality inspection information determination module is used to determine the reference quality inspection information required when inspecting the target retrieval tag information of the target image based on the retrieval tag quality inspection condition information. The reference quality inspection information includes at least one of the following: the target retrieval tag information of the target image, the target positioning information of the target image, and the first global image features and the second global image features extracted from the target image by the global feature extraction model before and after training using the retrieval tag information of the target image, respectively. The retrieval tag information quality inspection module is used to perform quality inspection on the retrieval tag information of the target image based on the reference quality inspection information; The retrieval tag information quality inspection module is specifically used to: perform quality inspection on the retrieval tag information of the target image using at least one of the following methods: The quality inspection method based on target scatter plots includes: generating target scatter plot information based on the target location included in the target positioning information of each target image, wherein the target location is used to describe the horizontal and vertical coordinates of the image acquisition location of the target image, and each scatter point in the target scatter plot information corresponds to one target image; marking the scatter points in the target scatter plot information based on the target retrieval tag information, the first global image feature, and the second global image feature of each target image, wherein the same scatter point in the target scatter plot information is marked using different marking operations, and each dimension of the reference quality inspection information corresponds to a marking operation; and performing quality inspection on the retrieval tag information of the target image based on the marking values ​​obtained by applying different marking operations to each scatter point in the target scatter plot information. The quality inspection method based on the difference scatter plot includes: generating difference scatter plot information based on the target positioning information of each target image, wherein each scatter point in the difference scatter plot information corresponds to a pair of target images, and each scatter point in the difference scatter plot information is determined by the positional and angular differences between the pair of target images; marking the scatter points in the difference scatter plot information based on the first global image features and the second global image features of each target image, wherein the marking value of the scatter point is determined by the similarity of the global image features between the pair of target images corresponding to the scatter point; and performing quality inspection on the retrieval tag information of the target images based on the marking values ​​of each scatter point in the difference scatter plot information. The quality inspection method based on mean and variance includes: determining the mean and variance of the location information of multiple categories of images based on the target retrieval tag information of each target image, the first global image feature, and the second global image feature. The mean and variance of the location information of each category of images includes values ​​determined by different calculation methods, and each dimension of the data in the reference quality inspection information corresponds to a calculation method; and performing quality inspection on the retrieval tag information of the target image based on the mean and variance of the location information of multiple categories of images determined by different calculation methods. The similarity-based quality inspection method includes: determining a retrieval target image closest to the cluster center from the target images of each category label based on the target category label information included in the target retrieval label information of each target image; determining the second global image feature of the retrieval target image; constructing retrieval data by combining the second global image feature of the retrieval target image with the target positioning information of the retrieval target image; using the remaining target images of each category label as query target images, and determining the second global image feature of the query target images; calculating the similarity between the query target images and all retrieval target images in the retrieval data, in order to perform quality inspection on the retrieval label information of the target images.

14. An image retrieval tag quality inspection electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the image retrieval tag quality inspection method as described in any one of claims 1-12.

15. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the image retrieval tag quality inspection method as described in any one of claims 1-12.

Images