Image retrieval sample registration method and apparatus, electronic device and medium

Abstract

An image retrieval sample registration method and apparatus, an electronic device and a medium. The method comprises: acquiring an original image sample and a category label corresponding to the original image sample; performing different transformation processing on the original image sample to obtain a plurality of transformed images; on the basis of a preset feature extraction mode, extracting image features of each image retrieval sample, the image retrieval samples comprising the original image sample and the plurality of transformed images; and storing the image features and corresponding category label of each image retrieval sample. As the transformed images are obtained by performing transformation processing on the original image sample, and the transformation processing will not change the category of an image, the transformed images and the original image sample can be used as the image retrieval samples of the same category, such that the image retrieval samples can relatively completely reflect the feature diversity of the samples of the same category. In this way, the number of image retrieval samples can be increased, improving the accuracy of image retrieval.

Description

Image retrieval sample registration method, device, electronic device and medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on April 2, 2024, with application number 202410397299.2 and invention name “A registration method, device, electronic device and medium for image retrieval samples”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of image retrieval technology, and in particular to a registration method, device, electronic device, and medium for image retrieval samples. Background Art

[0003] Image classification algorithms based on image retrieval primarily involve two processes: registration and retrieval. During the registration process, image retrieval samples with known labels are obtained, their image features are extracted, and a database is established to store the relationship between each label and image feature. After the image database is registered, the electronic device can retrieve images with unknown labels. Specifically, the image features of the image to be retrieved are extracted and compared with the image features stored in the image database to determine the retrieval results.

[0004] The above retrieval results are determined based on the comparison of image features. Therefore, the quantity and quality of image retrieval samples are crucial to the accuracy of retrieval results. Some image retrieval samples are expensive to generate, making it difficult to obtain a sufficient number of image retrieval samples in a short period of time in certain situations. When image retrieval samples are limited, the registered image retrieval samples cannot fully reflect the feature diversity of samples within the same category, nor can they clearly identify the key differences between samples of different categories. This can lead to misclassification of searched images and low image retrieval accuracy. Summary of the Invention

[0005] The purpose of the embodiments of this application is to provide a method, device, electronic device, and medium for registering image retrieval samples to expand the number of image retrieval samples and improve the accuracy of image retrieval. The specific technical solutions are as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for registering an image retrieval sample, the method comprising:

[0007] Obtaining an original image sample and a category label corresponding to the original image sample;

[0008] Performing different transformation processes on the original image samples to obtain multiple transformed images;

[0009] Extracting image features of each image retrieval sample according to a preset feature extraction method, wherein the image retrieval sample includes the original image sample and the multiple transformed images, and any two images included in the image retrieval sample are not completely identical;

[0010] The image features and corresponding category labels of each image retrieval sample are stored for use in image retrieval.

[0011] Optionally, the step of performing different transformation processes on the original image samples to obtain a plurality of transformed images includes:

[0012] For each original image sample, at least one of the following preset processing methods is used to transform the original image sample to obtain multiple transformed images, wherein the preset processing methods include: region of interest disturbance processing, brightness transformation processing, and pixel transformation processing.

[0013] Optionally, the method of performing disturbance processing on the region of interest includes:

[0014] For each original image sample, determining a position of a region of interest of the original image sample;

[0015] Moving at least one side of the region of interest position by a target distance along a target direction, and using the intersection of a straight line on which the moved side lies and a straight line on which its adjacent side lies as a vertex of the region of interest, to obtain a modified region of interest position, wherein the target direction is the direction of a straight line on which the adjacent side of the side to be moved lies, and the ratio of the target distance to the length of the adjacent side of the side to be moved is a target ratio; and / or rotating the region of interest position by a target angle to obtain a modified region of interest position;

[0016] The original image sample is cropped according to the changed position of the region of interest to obtain a transformed image.

[0017] Optionally, the step of determining the position of the region of interest of the original image sample includes:

[0018] If the region of interest information of the original image sample is obtained, the minimum bounding rectangle of the region of interest is calculated according to the region of interest information as the position of the region of interest of the original image sample, wherein the region of interest information is used to identify the position of the region of interest; or

[0019] If the region of interest information of the original image sample is not obtained, the entire region of the original image sample is determined as the region of interest position.

[0020] Optionally, before the step of moving at least one side of the region of interest by a target distance along a target direction, the method further includes:

[0021] Obtaining a preset ratio range, wherein the preset ratio range is predetermined based on a disturbance degree of the region of interest;

[0022] A corresponding target ratio is determined for each edge from the preset ratio range.

[0023] Optionally, the preset feature extraction method includes multiple feature extraction methods;

[0024] After the step of storing the image features and corresponding category labels of each image retrieval sample, the method further includes:

[0025] For each preset feature extraction method, calculate the intra-class similarity between target image features corresponding to image retrieval samples of the same category, and calculate the inter-class similarity between target image features corresponding to image retrieval samples of different categories, wherein the target image features are image features extracted from the image retrieval samples using the preset feature extraction method;

[0026] Based on the intra-class similarity and the inter-class similarity, determine the method weight corresponding to the preset feature extraction method, so that the intra-class similarity is positively correlated with the method weight, and the inter-class similarity is negatively correlated with the method weight; or, calculate the difference between the intra-class similarity and the inter-class similarity corresponding to the preset feature extraction method; based on the difference, determine the method weight corresponding to the preset feature extraction method, so that the difference is positively correlated with the method weight.

[0027] Optionally, after the step of determining a method weight corresponding to the preset feature extraction method based on the intra-class similarity and the inter-class similarity, the method further includes:

[0028] Get the image to be retrieved;

[0029] extracting a plurality of image features to be retrieved of the image to be retrieved respectively according to the plurality of feature extraction methods;

[0030] Calculating the similarity between each image feature to be retrieved and the image feature to be compared of each image retrieval sample, wherein the feature extraction method of the image feature to be compared is the same as the feature extraction method of the image feature to be retrieved;

[0031] The image retrieval results are determined based on the similarity and method weight corresponding to each image feature to be retrieved.

[0032] Optionally, the step of determining the image retrieval result based on the similarity and mode weight corresponding to each image feature to be retrieved includes:

[0033] For each image retrieval sample, each similarity corresponding to the image retrieval sample is weighted and summed according to the corresponding method weight to obtain a weighted summation result, wherein the method weight corresponding to each similarity is the weight corresponding to the feature extraction method of the image feature to be retrieved corresponding to the similarity;

[0034] The category to which the image retrieval sample with the highest corresponding weighted result belongs is determined as the image retrieval result corresponding to the image to be retrieved.

[0035] Optionally, the image feature is a matrix;

[0036] After the step of storing the image features and corresponding category labels of each image retrieval sample, the method further includes:

[0037] For each image feature, calculate the intra-class similarity between the eigenvalues ​​of the target dimension in the image feature corresponding to the image retrieval samples of the same category, and the inter-class similarity between the eigenvalues ​​of the target dimension in the image feature corresponding to the image retrieval samples of different categories, wherein the target dimension is each row or column of the matrix;

[0038] Based on the intra-class similarity and the inter-class similarity, a dimension weight corresponding to the target dimension is determined, so that the intra-class similarity is positively correlated with the dimension weight, and the inter-class similarity is negatively correlated with the dimension weight.

[0039] Optionally, after the step of determining the dimension weight corresponding to the target dimension based on the intra-class similarity and the inter-class similarity, the method further includes:

[0040] Get the image to be retrieved;

[0041] Extracting the image features to be retrieved of the image to be retrieved according to the preset feature extraction method;

[0042] Calculating the similarity between the feature value of each dimension of the image feature to be retrieved and the feature value of the corresponding dimension of the image feature to be compared of each image retrieval sample, wherein the feature extraction method of the image feature to be compared is the same as the feature extraction method of the image feature to be retrieved;

[0043] An image retrieval result is determined based on the similarity corresponding to the feature value of each dimension in the image features to be retrieved and the dimension weight.

[0044] Optionally, the step of determining the image retrieval result based on the similarity corresponding to the feature value of each dimension in the image feature to be retrieved and the dimension weight includes:

[0045] For each image retrieval sample, each similarity corresponding to the image retrieval sample is weighted and summed according to the corresponding dimension weight to obtain a weighted summation result, wherein the dimension weight corresponding to each similarity is the weight corresponding to the dimension of the image feature to be retrieved corresponding to the similarity;

[0046] The category to which the image retrieval sample with the highest corresponding weighted result belongs is determined as the image retrieval result corresponding to the image to be retrieved.

[0047] In a second aspect, an embodiment of the present application provides a device for registering image retrieval samples, the device comprising:

[0048] An image acquisition module is used to acquire original image samples and category labels corresponding to the original image samples;

[0049] A transformation processing module, configured to perform different transformation processes on the original image samples to obtain a plurality of transformed images;

[0050] a feature extraction module, configured to extract image features of each image retrieval sample according to a preset feature extraction method, wherein the image retrieval sample includes the original image sample and the plurality of transformed images, and any two images included in the image retrieval sample are not completely identical;

[0051] The feature storage module is used to store the image features and corresponding category labels of each image retrieval sample for use in image retrieval.

[0052] Optionally, the transformation processing module includes:

[0053] The transformation processing submodule is used to transform each original image sample using at least one of the following preset processing methods to obtain multiple transformed images, wherein the preset processing methods include: region of interest perturbation processing, brightness transformation processing, and pixel transformation processing.

[0054] Optionally, the transformation processing submodule includes:

[0055] a position determining unit, configured to determine, for each original image sample, a position of a region of interest of the original image sample;

[0056] a position changing unit, configured to move at least one side of the position of the region of interest by a target distance along a target direction, and use the intersection of a straight line on which the moved side is located and a straight line on which its adjacent side is located as a vertex of the region of interest, to obtain a changed position of the region of interest, wherein the target direction is the direction of a straight line on which the adjacent side of the side to be moved is located, and the ratio of the target distance to the length of the adjacent side of the side to be moved is a target ratio;

[0057] A region rotation unit, configured to rotate the position of the region of interest by a target angle to obtain a changed position of the region of interest;

[0058] The image cropping unit is used to crop the original image sample according to the changed position of the region of interest to obtain a transformed image.

[0059] Optionally, the position determination unit includes:

[0060] a first position determining subunit, configured to, if information about the region of interest of the original image sample is obtained, calculate a minimum bounding rectangle of the region of interest according to the information about the region of interest as the position of the region of interest of the original image sample, wherein the information about the region of interest is used to identify the position of the region of interest;

[0061] The second position determining subunit is configured to determine the entire area of ​​the original image sample as the position of the region of interest if the region of interest information of the original image sample is not acquired.

[0062] Optionally, the device further includes:

[0063] A range acquisition module, configured to acquire a preset ratio range, wherein the preset ratio range is predetermined based on a disturbance degree of a region of interest;

[0064] a ratio determination module, configured to determine a corresponding target ratio for each edge from the preset ratio range;

[0065] The preset feature extraction method includes multiple feature extraction methods.

[0066] Optionally, the device further includes:

[0067] an intra-class similarity calculation module, configured to calculate, for each preset feature extraction method, the intra-class similarity between target image features corresponding to image retrieval samples of the same category, and to calculate the inter-class similarity between target image features corresponding to image retrieval samples of different categories, wherein the target image features are image features extracted from the image retrieval samples using the preset feature extraction method;

[0068] The first weight determination module is used to determine the method weight corresponding to the preset feature extraction method based on the intra-class similarity and the inter-class similarity, so that the intra-class similarity is positively correlated with the method weight, and the inter-class similarity is negatively correlated with the method weight; or, calculate the difference between the intra-class similarity and the inter-class similarity corresponding to the preset feature extraction method; based on the difference, determine the method weight corresponding to the preset feature extraction method, so that the difference is positively correlated with the method weight.

[0069] Optionally, the device further includes:

[0070] A first image acquisition module, used to acquire an image to be retrieved;

[0071] A first feature extraction module is used to extract multiple image features to be retrieved of the image to be retrieved according to the multiple feature extraction methods;

[0072] an inter-class similarity calculation module, configured to calculate the similarity between each image feature to be retrieved and the image feature to be compared of each image retrieval sample, wherein the feature extraction method of the image feature to be compared is the same as the feature extraction method of the image feature to be retrieved;

[0073] The first result determination module is used to determine the image retrieval result based on the similarity and method weight corresponding to each image feature to be retrieved.

[0074] Optionally, the first result determination module includes:

[0075] A first calculation submodule is configured to perform weighted summation on each similarity corresponding to each image retrieval sample according to a corresponding method weight, to obtain a weighted summation result, wherein the method weight corresponding to each similarity is the weight corresponding to the feature extraction method of the image feature to be retrieved corresponding to the similarity;

[0076] The first result determination submodule is configured to determine the category to which the image retrieval sample with the highest corresponding weighted result belongs as the image retrieval result corresponding to the image to be retrieved.

[0077] Optionally, the image feature is a matrix;

[0078] The device further comprises:

[0079] a third similarity calculation module, configured to calculate, for each image feature, an intra-class similarity between eigenvalues ​​of a target dimension in the image feature corresponding to image retrieval samples of the same category, and an inter-class similarity between eigenvalues ​​of the target dimension in the image feature corresponding to image retrieval samples of different categories, wherein the target dimension is each row or column of the matrix;

[0080] The second weight determination module is used to determine the dimension weight corresponding to the target dimension based on the intra-class similarity and the inter-class similarity, so that the intra-class similarity is positively correlated with the dimension weight, and the inter-class similarity is negatively correlated with the dimension weight.

[0081] Optionally, the device further includes:

[0082] A second image acquisition module, used to acquire the image to be retrieved;

[0083] A second feature extraction module is used to extract the image features to be retrieved of the image to be retrieved according to the preset feature extraction method;

[0084] a fourth similarity calculation module, configured to calculate the similarity between the feature value of each dimension of the image feature to be retrieved and the feature value of the corresponding dimension of the image feature to be compared of each image retrieval sample, wherein the feature extraction method of the image feature to be compared is the same as the feature extraction method of the image feature to be retrieved;

[0085] The second result determination module is used to determine the image retrieval result based on the similarity corresponding to the feature value of each dimension in the image feature to be retrieved and the dimension weight.

[0086] Optionally, the second result determination module includes:

[0087] The second calculation submodule is used to perform weighted summation on each similarity corresponding to each image retrieval sample according to the corresponding dimension weight to obtain a weighted summation result, wherein the dimension weight corresponding to each similarity is the weight corresponding to the dimension of the image feature to be retrieved corresponding to the similarity;

[0088] The second result determination submodule is configured to determine the category to which the image retrieval sample with the highest corresponding weighted result belongs as the image retrieval result corresponding to the image to be retrieved.

[0089] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0090] Memory for storing computer programs;

[0091] The processor is configured to implement any of the methods described in the first aspect above when executing a program stored in the memory.

[0092] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements any of the methods described in the first aspect above.

[0093] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when run on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0094] Beneficial effects of the embodiments of the present application:

[0095] In the solution provided by the embodiment of the present application, an electronic device can obtain an original image sample and a category label corresponding to the original image sample; perform different transformations on the original image sample to obtain multiple transformed images; extract the image features of each image retrieval sample according to a preset feature extraction method, wherein the image retrieval sample includes the original image sample and multiple transformed images, and any two images included in the image retrieval sample are not exactly the same; store the image features of each image retrieval sample and the corresponding category label for use in image retrieval. Since the transformed image is obtained by transforming the original image sample, and the transformation does not change the category to which the image belongs, the transformed image and the original image sample can be used as image retrieval samples of the same category. Even if the number of original image samples is limited, the original image sample can be transformed to obtain multiple transformed images, which can enable the image retrieval sample to more completely reflect the feature diversity of samples of the same category and clarify the key difference features between samples of different categories. In this way, the number of image retrieval samples can be expanded and the accuracy of image retrieval can be improved.

[0096] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0098] FIG1 is a schematic diagram of a process of image retrieval in the current related art;

[0099] FIG2 is a flow chart of a registration method for an image retrieval sample provided in an embodiment of the present application;

[0100] FIG3( a ) is a schematic diagram of retrieving samples of multiple images of the same category based on the embodiment shown in FIG2 ;

[0101] FIG3( b ) is a schematic diagram of retrieving samples of multiple images of different categories based on the embodiment shown in FIG2 ;

[0102] FIG4 is a flow chart of performing a region of interest disturbance process on an original image sample based on the embodiment shown in FIG2 ;

[0103] FIG5 is a schematic diagram of a process of changing the position of a region of interest based on the embodiment shown in FIG2 ;

[0104] FIG6 is a specific flow chart of step S401 in the embodiment shown in FIG4 ;

[0105] FIG7 is a flow chart of a method for determining a target ratio based on the embodiment shown in FIG2 ;

[0106] FIG8 is a flow chart of a method for calculating mode weights based on the embodiment shown in FIG2 ;

[0107] FIG9 is a flow chart of a method for determining image retrieval results based on the embodiment shown in FIG8 ;

[0108] FIG10 is a specific flow chart of step S904 in the embodiment shown in FIG9 ;

[0109] FIG11 is a flow chart of a method for calculating dimension weights based on the embodiment shown in FIG2 ;

[0110] FIG12 is a flow chart of a method for determining image retrieval results based on the embodiment shown in FIG11 ;

[0111] FIG13 is a specific flow chart of step S1204 in the embodiment shown in FIG12 ;

[0112] FIG14 is a schematic diagram of an image retrieval sample registration method based on the embodiment shown in FIG2 ;

[0113] FIG15 is a schematic diagram of the structure of a registration device for image retrieval samples provided in an embodiment of the present application;

[0114] FIG16 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0115] To make the objectives, technical solutions, and advantages of this application more clearly understood, the present application is further described below with reference to the accompanying drawings and examples. It is apparent that the described examples are only a portion of the embodiments of this application, and not all of them. All other embodiments derived by persons of ordinary skill in the art based on the examples in this application are intended to fall within the scope of protection of this application.

[0116] In the current related technology, the image retrieval process diagram can be shown in Figure 1. The left half of the dotted line is the sample image registration process, and the right half of the dotted line is the image retrieval process. Specifically, it includes the following steps:

[0117] S101, extracting features from the original image sample;

[0118] During the original image sample registration process, the electronic device may perform feature extraction on original image sample 1, original image sample i, and original image sample n to obtain image features of the n original image samples. The image features of the n original image samples and corresponding category labels are then stored in a database.

[0119] S102, extracting features from the test image;

[0120] During the image retrieval process, the electronic device can perform feature extraction on the test image without a category label to obtain image features of the test image. The feature extraction method used in step S102 is the same as the feature extraction method used in step S101.

[0121] S103, image retrieval.

[0122] After obtaining the image features of the test image, the electronic device can calculate the similarity between the image features of the test image and the image features of each original image sample stored in the database. The category of the original image sample with the highest similarity is determined as the category of the test image, thus obtaining the search result.

[0123] As can be seen, in current related technologies, the image features used for image retrieval are derived from feature extraction of raw image samples. If the number of raw image samples is limited, the registered raw image samples will not fully reflect the feature diversity of samples of the same category, nor will they be able to clearly identify the key differences between samples of different categories, resulting in low image retrieval accuracy.

[0124] In order to expand the number of image retrieval samples and improve the accuracy of image retrieval, the embodiments of the present application provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for registering image retrieval samples. The following first introduces a method for registering image retrieval samples provided by the embodiments of the present application.

[0125] The image retrieval sample registration method provided in the embodiments of this application can be applied to any electronic device that needs to retrieve images. For example, it can be an image retrieval device, an image retrieval system, a processing device, etc., without specific limitation here. For clarity of description, it will be referred to as an electronic device in this article.

[0126] As shown in FIG2 , a method for registering an image retrieval sample includes:

[0127] S201, obtaining an original image sample and a category label corresponding to the original image sample;

[0128] S202, performing different transformation processes on the original image samples to obtain multiple transformed images;

[0129] S203, extracting image features of each image retrieval sample according to a preset feature extraction method;

[0130] The image retrieval sample includes the original image sample and the multiple transformed images, and any two images included in the image retrieval sample are not completely the same.

[0131] S204: Store the image features and corresponding category labels of each image retrieval sample for use in image retrieval.

[0132] It can be seen that in the embodiment of the present application, the electronic device can obtain the original image sample and the category label corresponding to the original image sample; perform different transformation processing on the original image sample to obtain multiple transformed images; extract the image features of each image retrieval sample according to the preset feature extraction method, wherein the image retrieval sample includes the original image sample and multiple transformed images, and any two images included in the image retrieval sample are not exactly the same; store the image features of each image retrieval sample and the corresponding category label for image retrieval. Since the transformed image is obtained by transforming the original image sample, and the transformation processing does not change the category to which the image belongs, the transformed image and the original image sample can be used as image retrieval samples of the same category. Even if the number of original image samples is limited, the original image sample can be transformed to obtain multiple transformed images, which can make the image retrieval sample more completely reflect the feature diversity of samples of the same category and clarify the key difference features between samples of different categories. In this way, the number of image retrieval samples can be expanded and the accuracy of image retrieval can be improved.

[0133] In step S201, the electronic device may obtain an original image sample and a category label corresponding to the original image sample. For example, assuming that the original image sample 1 is a vehicle image, the electronic device may obtain the original image sample 1 and the category label "vehicle" corresponding to the original image sample 1.

[0134] Since transforming the original image sample does not change the category to which the image belongs, if the number of images corresponding to a certain category is limited, the electronic device can perform different transformations on the original image sample to obtain multiple transformed images, thus executing step S202. Furthermore, the original image sample and the multiple transformed images are used as image retrieval samples. Since the multiple transformed images are obtained by performing different transformations on the original image sample, no two images included in the image retrieval sample are exactly the same.

[0135] The above-mentioned transformation processing may be at least one of a region of interest disturbance processing, a brightness transformation processing, and a pixel transformation processing, which is not specifically limited here.

[0136] For example, the original image samples can be subjected to ROI perturbation processing to obtain transformed image 1; the original image samples can be subjected to brightness transformation processing to obtain transformed image 2; the original image samples can be subjected to pixel transformation processing and ROI perturbation processing in sequence to obtain transformed image 3.

[0137] After expanding the image retrieval samples, the generated multiple transformed images can more comprehensively reflect the feature diversity of samples of the same category and clarify the key differences between samples of different categories. A schematic diagram of multiple image retrieval samples of the same category can be shown in Figure 3(a). The two image retrieval samples in Figure 3(a) have essentially the same texture and belong to the same category. The two image retrieval samples are obtained by perturbing the same original sample image using a region of interest. Specifically, the left image is obtained by taking a circular region of interest from the original sample image, and the right image is obtained by taking a rectangular region of interest from the original sample image, which can reflect the feature diversity of the category. A schematic diagram of image retrieval samples of different categories can be shown in Figure 3(b). The four image retrieval samples in Figure 3(b) belong to different categories. The textures of the four image retrieval samples are different, which can reflect the differences between different categories.

[0138] If the image features of the image to be retrieved are highly similar to the image features of the image retrieval samples in a certain category, then it can be determined that the image to be retrieved belongs to that category. In order to determine the features of the image retrieval samples in each category and then perform image retrieval based on the features of the image retrieval samples, the electronic device can extract the image features of each image retrieval sample according to a preset feature extraction method, that is, execute step S203.

[0139] Among them, the preset feature extraction method can be used to extract deep learning features, global features or local features of the image. The above-mentioned global features may include LBP (Local Binary Patterns) features, HOG (Histogram of Oriented Gradient) features, and grayscale histogram features. The above-mentioned local features may include SIFT (Scale Invariant Feature Transform) features, Harris (corner detection) features, ORB (Oriented FAST and Rotated BRIEF, feature point detection) features, all of which are not specifically limited here. For example, the electronic device can extract the HOG features of the image retrieval sample, or it can extract the SIFT features of the image retrieval sample.

[0140] As an implementation method, the dimensions of the image features extracted by the electronic device are all equal, so as to facilitate the subsequent calculation of the similarity between the image features.

[0141] After extracting the image features of each image retrieval sample, the features corresponding to the category to which the image retrieval sample belongs can be reflected based on the image features of the image retrieval sample. To perform image retrieval, the electronic device can store the image features of each image retrieval sample and the corresponding category label, that is, execute step S204. Specifically, the electronic device can store the image features of each image retrieval sample and the corresponding category label in a database, cloud, or storage device, etc., which is not specifically limited here.

[0142] For example, assuming that the category label corresponding to image retrieval sample 1-image retrieval sample 3 is category A, and the image features corresponding to image retrieval sample 1-image retrieval sample 3 are image feature 1-image feature 3, the electronic device can store the correspondence between category A and image features 1-image feature 3 in the database.

[0143] If the above correspondence is stored in a database, the categories in the database and the number and specific content of the image features corresponding to each category can be changed. The constructed database allows operations such as addition, deletion, and query. In other words, after the above correspondence is stored in the database, the user can delete or add image features in the database according to actual needs. They can also query the database to determine the category to which a certain image feature belongs, or query the image features corresponding to a certain category.

[0144] As an implementation method, the j-th image feature of the i-th category can be denoted as F (i,j) .

[0145] As can be seen, in the embodiments of the present application, since the transformed image is obtained by transforming the original image sample, and the transformation process does not change the category to which the image belongs, the transformed image and the original image sample can be used as image retrieval samples of the same category. Even if the number of original image samples is limited, the original image samples can be transformed to obtain multiple transformed images. This allows the image retrieval samples to more completely reflect the feature diversity of samples of the same category and to clearly identify the key differences between samples of different categories. In this way, the number of image retrieval samples can be expanded, and the accuracy of image retrieval can be improved.

[0146] As an implementation of an embodiment of the present application, the step of performing different transformations on the original image samples to obtain multiple transformed images may include:

[0147] For each original image sample, at least one of the following preset processing methods is used to transform the original image sample to obtain multiple transformed images, wherein the preset processing methods include: region of interest disturbance processing, brightness transformation processing, and pixel transformation processing.

[0148] Because transforming the original image samples does not change the image category, in order to obtain an image that belongs to the same category as the original image samples but has certain differences from the original image samples, the electronic device can transform each original image sample using at least one of the preset processing methods to obtain multiple transformed images.

[0149] The preset processing methods may include ROI perturbation processing, brightness conversion processing, and pixel conversion processing. The brightness conversion processing may include gamma correction processing, contrast enhancement processing, linear stretching processing, etc.; the pixel conversion processing may include Gaussian filtering processing, mean filtering processing, motion blur processing, noise addition processing, etc., all of which are not specifically limited here.

[0150] In one embodiment, there is one original image sample I. The electronic device can transform the original image sample I using at least one of the preset processing methods to obtain an image group consisting of n transformed images {I1, I2, ..., I n For example, the electronic device may respectively perform transformation processing on the original image sample by using contrast enhancement processing, mean filtering processing, and motion blur processing to obtain transformed images 1 to 3.

[0151] In another embodiment, there are multiple original image samples. For multiple original image samples, the electronic device can transform the original image sample using at least one of the preset processing methods to obtain multiple transformed images. For example, there are two original image samples, namely original image sample 1 and original image sample 2. The electronic device can transform the original image sample 1 using Gaussian filtering, contrast enhancement, and linear stretching to obtain transformed images 1 to 3; the electronic device can transform the original image sample 2 using noise addition and region of interest perturbation to obtain transformed images 4 to 5.

[0152] It can be seen that in the embodiment of the present application, the electronic device can transform the original image sample using at least one of the following preset processing methods for each original image sample to obtain multiple transformed images, wherein the preset processing methods include: region of interest disturbance processing, brightness transformation processing, and pixel transformation processing. Since the transformation processing of the original image sample does not change the category to which the image belongs. Therefore, in order to obtain an image that is of the same category as the original image sample and has a certain difference from the original image sample, the electronic device can transform the original image sample using at least one of the preset processing methods for each original image sample to obtain multiple transformed images. In this way, the number of image retrieval samples can be expanded, thereby more completely reflecting the feature diversity of samples of the same type, and clarifying the key difference features between samples of different categories, thereby improving the accuracy of image retrieval.

[0153] As an implementation of an embodiment of the present application, as shown in FIG4 , the above-mentioned method of perturbation processing of the region of interest may include:

[0154] S401, for each original image sample, determining a position of a region of interest of the original image sample;

[0155] Since the region of interest in an image can reflect the characteristics of the image, the electronic device can determine the location of the region of interest for each original image sample. The shape of the region of interest can be rectangular, pentagonal, hexagonal, circular, etc., which is not specifically limited here.

[0156] For example, the electronic device may input the original image sample into a machine vision model to obtain the region of interest of the original image sample output by the machine vision model. For another example, the region of interest of the original image sample may be pre-calibrated, and the electronic device may directly obtain the pre-calibrated region of interest of the original image sample.

[0157] S402: moving at least one side of the region of interest by a target distance in a target direction, and using the intersection of a straight line containing the moved side and a straight line containing an adjacent side as a vertex of the region of interest to obtain a modified region of interest position; and / or rotating the region of interest position by a target angle to obtain a modified region of interest position;

[0158] Since the image features of the region of interest will also change after the region of interest changes, in order to obtain a transformed image that is somewhat different from the original image sample, the region of interest can be modified based on any of the following three implementations.

[0159] In a first embodiment, the electronic device can move at least one side of the region of interest by a target distance in a target direction. Since one or more sides of the region of interest can change the region of interest position after being moved, at least one side of the region of interest position can be moved. Since after a side of the region of interest is moved, the side may no longer intersect with an adjacent side, and the region of interest is no longer closed. Therefore, in order to ensure that the region of interest remains closed after the move, the intersection of the straight line containing the moved side and the straight line containing the adjacent side can be used as the vertex of the region of interest to obtain the changed region of interest position. In this way, the image features corresponding to the changed region position will differ from the image features corresponding to the region of interest.

[0160] The target direction is the direction of the line adjacent to the side to be moved. For example, if the region of interest is a rectangle, and the side to be moved is the left or right side of the rectangle, the target direction is the direction of the line adjacent to the top or bottom side. If the side to be moved is the top or bottom side of the rectangle, the target direction is the direction of the line adjacent to the left or right side.

[0161] For example, a schematic diagram of the process of changing the position of the region of interest can be shown in Figure 5. The side to be moved is the upper side of the position of the region of interest. The electronic device can move the upper side upward by a target distance r along the direction of the straight line on the left or right side. The state of the position of the region of interest is shown in the left figure. At this time, there is no longer any intersection between the upper side after the move and the left and right sides. In order to make the changed region of interest still closed, the first intersection 501 of the straight line where the upper side is located after the move and the straight line where the left side is located, and the second intersection 502 of the straight line where the upper side is located after the move and the straight line where the right side is located can be determined. The state of the position of the region of interest is shown in the middle figure. Next, the first intersection 501 and the second intersection 502 can be used as the vertices of the region of interest to obtain the changed position of the region of interest. The state of the position of the region of interest is shown in the right figure.

[0162] In the first case, the target ratio is the ratio of the target distance to the length of the adjacent side to be moved. Thus, the electronic device can calculate the target distance corresponding to the side to be moved and then move the side in the target direction by the target distance. The target ratio can be pre-set, specifically 10%, 20%, 30%, etc., and is not specifically limited here.

[0163] For example, the region of interest is a rectangle. The sides to be moved are the left and bottom sides of the region of interest. The lengths of the top and bottom sides of the region of interest are 10 cm, and the lengths of the left and right sides are 5 cm. The target ratios corresponding to the left and bottom sides are 10% and 20%, respectively. Therefore, the target distances corresponding to the left and bottom sides can be calculated to be 0.5 cm and 2 cm, respectively. Then, the left side is moved 0.5 cm along the line containing the top or bottom side, and the bottom side is moved 2 meters along the line containing the left or right side. The intersection points of the moved bottom side with the lines containing the left and right sides, and the intersection points of the left side with the line containing the top side, are used as the vertices of the region of interest.

[0164] In the second case, the ratio of the area change of the region of interest before and after the change to the area of ​​the region of interest before the change is the target ratio. In this case, the electronic device can move the side to be moved a certain distance along the target direction so that the area change of the region of interest before and after the change is the product of the target ratio and the area of ​​the region of interest before the change. For example, the region of interest is a rectangle. The side to be moved is the top side, and the target ratio corresponding to the top side is 5%. Assume that the area of ​​the region of interest before the change is 100cm 2 , then the upper edge can be moved a certain distance along the left or right straight line so that the area change of the region of interest before and after the change is 5cm 2 .

[0165] In the third case, the ratio of the target distance to the preset distance is the target ratio described above. In this case, the electronic device can calculate the target distance and then move the edge to be moved by the target distance in the target direction. For example, the region of interest is a rectangle. The edge to be moved is the top edge of the region of interest, and the corresponding target ratio is 4%. The preset distance is 50 cm. The target distance corresponding to the top edge can be calculated to be 2 cm. The top edge of the rectangle can then be moved 2 cm along the line on the left or right side.

[0166] In a second embodiment, in order to change the position of the region of interest, the electronic device may rotate the position of the region of interest by a target angle to obtain a changed position of the region of interest. Since the image features corresponding to the rotated region of interest will change compared to the image features corresponding to the region of interest before the rotation, in order to obtain a transformed image that is somewhat different from the original image sample, the electronic device may rotate the position of the region of interest by a target angle to obtain a changed position of the region of interest. The target angle may be 5°, 30°, 60°, etc., and is not specifically limited here. For example, the electronic device may rotate the position of the region of interest by 45° clockwise to obtain a changed position of the region of interest. For another example, the electronic device may rotate the position of the region of interest by 15° counterclockwise to obtain a changed position of the region of interest.

[0167] In a third embodiment, the electronic device can modify the region of interest in combination with the first and second embodiments. For example, the region of interest is first modified based on the first embodiment, and then the modified region of interest position is modified based on the second embodiment to obtain a modified region of interest position. Specifically, the electronic device can move at least one side of the region of interest position by a target distance in a target direction, and use the intersection of the straight line containing the moved side and the straight line containing its adjacent side as the vertex of the region of interest to obtain the modified region of interest position. The region of interest position is then rotated by a target angle to obtain the modified region of interest position.

[0168] S403: Crop the original image sample according to the changed position of the region of interest to obtain a transformed image.

[0169] After determining the location of the region of interest, the electronic device can crop the original image sample according to the modified location of the region of interest to obtain a transformed image. In this way, the transformed image belongs to the same category as the original image sample, but is different, which can expand the number of image retrieval samples.

[0170] It can be seen that in the embodiment of the present application, the electronic device can determine the position of the region of interest of each original image sample; move the position of at least one side of the region of interest position by a target distance along a target direction, and use the intersection of the straight line where the moved side is located and the straight line where its adjacent side is located as the vertex of the region of interest to obtain the changed position of the region of interest, wherein the target direction is the direction of the straight line where the adjacent side of the side to be moved is located, and the ratio of the target distance to the length of the adjacent side of the side to be moved is the target ratio; and / or, after rotating the region of interest position by a target angle, obtain the changed position of the region of interest; crop the original image sample according to the changed position of the region of interest to obtain a transformed image. Since the transformed image is obtained by performing region change and / or rotation processing on the region of interest of the original image sample, the transformed image belongs to the same category as the original image sample, and the image features of the transformed image have certain differences from the image features of the original image sample. In this way, the number of image retrieval samples can be quickly expanded.

[0171] As an implementation of an embodiment of the present application, as shown in FIG6 , the step of determining the position of the region of interest of the original image sample may include:

[0172] S601, determining whether the region of interest information of the original image sample is obtained; if the region of interest information of the original image sample is obtained, executing step S602; if the region of interest information of the original image sample is not obtained, executing step S603;

[0173] In order to determine the region of interest of the original image sample, the electronic device may determine whether the region of interest information of the original image sample is acquired, and then execute step S602 or step S603.

[0174] S602, calculating the minimum bounding rectangle of the region of interest according to the region of interest information as the position of the region of interest of the original image sample;

[0175] When the ROI information of the original image sample is obtained, the electronic device can determine the ROI position identified by the ROI information, calculate the minimum bounding rectangle of the ROI position, and then use the minimum bounding rectangle as the ROI position of the original image sample.

[0176] For example, assuming that the ROI position identified by the ROI information of the original image sample is a pentagonal area, the electronic device can calculate the minimum bounding rectangle of the pentagonal area and use the minimum bounding rectangle as the ROI position of the original image sample.

[0177] S603: The entire area of ​​the original image sample is determined as the location of the region of interest.

[0178] In the case that the region of interest information of the original image sample is not obtained, the electronic device may determine the entire region of the original image sample as the location of the region of interest.

[0179] As can be seen, in the embodiments of the present application, if the ROI information of the original image sample is obtained, the electronic device can calculate the minimum bounding rectangle of the ROI based on the ROI information as the ROI location of the original image sample, where the ROI information is used to identify the location of the ROI. If the ROI information of the original image sample is not obtained, the entire area of ​​the original image sample is determined as the ROI location. In this way, the electronic device can quickly and accurately determine the ROI location of the original image sample.

[0180] As an implementation of an embodiment of the present application, as shown in FIG7 , before the step of changing the position of each side of the region of interest according to the target ratio to obtain the changed region position, the method may further include:

[0181] S701, obtaining a preset ratio range;

[0182] Before changing the position of each edge of the region of interest, a proportional range can be pre-set based on the degree of disturbance in the region of interest. To increase the degree of disturbance in the region of interest, the upper and lower limits of the preset proportional range can be set to be larger. For example, the upper and lower limits of the proportional range can be set to 10% and 20%, respectively, and the proportional range can be 10%-20%. To decrease the degree of disturbance in the region of interest, the upper and lower limits of the preset proportional range can be set to be smaller. For example, the upper and lower limits of the proportional range can be set to 0% and 10%, respectively, and the proportional range can be 0%-10%.

[0183] S702: Determine a corresponding target ratio for each edge from the preset ratio range.

[0184] After obtaining the preset ratio range, the electronic device can determine a corresponding target ratio for each edge within the preset ratio range. For example, assuming the ratio range is 10%-20%, the target ratio can be 12%, 15%, or 18%, etc. For another example, assuming the ratio range is 0%-10%, the target ratio can be 3%, 6%, or 9%, etc.

[0185] In the first case, the electronic device can determine a different target ratio for each edge. For example, if all four edges of the region of interest need to be moved, and the preset ratio range is 5%-10%, the target ratios for the top, bottom, left, and right edges can be determined to be 5.5%, 8%, 6%, and 9%, respectively.

[0186] In the second case, the electronic device may determine the same target ratio for each edge. For example, if the preset ratio range is 5%-10%, then the target ratio corresponding to each edge may be determined to be 7%.

[0187] In the third scenario, the electronic device can set the same target ratio for multiple edges and different target ratios for other edges. For example, if the preset ratio range is 5%-10%, the target ratios for the top and left edges can be set to 6%, and the target ratios for the bottom and right edges can be set to 8% and 7%, respectively. For another example, the electronic device can set the target ratios for the top, bottom, and right edges to 9%, and the target ratio for the left edge to 6%.

[0188] As can be seen, in the embodiments of the present application, the electronic device can obtain a preset ratio range, where the preset ratio range is pre-determined based on the disturbance level of the region of interest; and determine a corresponding target ratio for each edge from the preset ratio range. Because the ratio range can be pre-determined based on the disturbance level of the region of interest, the target ratio corresponding to each edge can be randomly determined within a controllable range. In this way, the target ratio can be quickly determined.

[0189] As an implementation of an embodiment of the present application, the above-mentioned preset feature extraction method includes multiple feature extraction methods. In this case, as shown in Figure 8, after the above-mentioned step of storing the image features of each image retrieval sample and the corresponding category label, the above-mentioned method may further include:

[0190] S801, for each preset feature extraction method, calculating the intra-class similarity between target image features corresponding to image retrieval samples of the same class, and calculating the inter-class similarity between target image features corresponding to image retrieval samples of different classes;

[0191] The electronic device may use a preset feature extraction method to extract features from the image retrieval samples to obtain target image features. If the similarity between the target image features corresponding to the image retrieval samples of the same category is high, and the similarity between the target image features corresponding to the image retrieval samples of different categories is low, it indicates that the characteristics of the image retrieval samples reflected by the target image features result in a high degree of similarity between the image retrieval samples of the same category and a low degree of similarity between the image retrieval samples of different categories.

[0192] In this way, the intra-class consistency of image retrieval samples of the same category and the inter-class differences of image retrieval samples of different categories can be better reflected, thereby improving the accuracy of image retrieval. Therefore, in order to determine a preset feature extraction method that makes the consistency evaluation of image retrieval samples of the same category higher and the consistency evaluation of image retrieval samples of different categories lower, the electronic device can calculate the intra-class similarity between the target image features corresponding to image retrieval samples of the same category, and calculate the inter-class similarity between the target image features corresponding to image retrieval samples of different categories for each preset feature extraction method.

[0193] For example, assume that image retrieval samples 1 to 3 belong to category A, and image retrieval samples 4 to 6 belong to category B. Use the preset feature extraction method corresponding to the HOG feature to extract features from image retrieval samples 1 to 6, and obtain HOG features 1 to HOG features 6; use the preset feature extraction method corresponding to the LBP feature to extract features from image retrieval samples 1 to 6, and obtain LBP features 1 to LBP features 6.

[0194] For the preset feature extraction method corresponding to the HOG feature, the electronic device can calculate the intra-class similarity between HOG feature 1 and HOG feature 3 corresponding to image retrieval sample 1 and image retrieval sample 3 belonging to category A, and the intra-class similarity between HOG feature 4 and HOG feature 6 corresponding to image retrieval sample 4 and image retrieval sample 6 belonging to category B. The intra-class similarities between HOG feature 1 and HOG feature 2, between HOG feature 1 and HOG feature 3, and between HOG feature 2 and HOG feature 3 are 86%, 92%, and 89%, respectively. The intra-class similarities between HOG feature 4 and HOG feature 5, between HOG feature 4 and HOG feature 6, and between HOG feature 5 and HOG feature 6 are 91%, 93%, and 87%, respectively.

[0195] The electronic device can calculate the inter-class similarity between HOG feature 1-HOG feature 3 corresponding to image retrieval sample 1-image retrieval sample 3 belonging to category A and HOG feature 4-HOG feature 6 corresponding to image retrieval sample 4-image retrieval sample 6 belonging to category B. The inter-class similarities between HOG feature 1 and HOG feature 4-HOG feature 6 are 3%, 5%, and 2% respectively; the inter-class similarities between HOG feature 2 and HOG feature 4-HOG feature 6 are 5%, 6%, and 1% respectively; and the inter-class similarities between HOG feature 3 and HOG feature 4-HOG feature 6 are 6%, 2%, and 3% respectively.

[0196] For the preset feature extraction method corresponding to the LBP feature, the electronic device can calculate the intra-class similarity between LBP feature 1 and LBP feature 3 corresponding to image retrieval sample 1 and image retrieval sample 3 belonging to category A, and the intra-class similarity between LBP feature 4 and LBP feature 6 corresponding to image retrieval sample 4 and image retrieval sample 6 belonging to category B. The intra-class similarities between LBP feature 1 and LBP feature 2, between LBP feature 1 and LBP feature 3, and between LBP feature 2 and LBP feature 3 are 23%, 15%, and 36%, respectively. The intra-class similarities between LBP feature 4 and LBP feature 5, between LBP feature 4 and LBP feature 6, and between LBP feature 5 and LBP feature 6 are 12%, 14%, and 9%, respectively.

[0197] The electronic device can calculate the inter-class similarity between LBP features 1-LBP features 3 corresponding to image retrieval samples 1-image retrieval samples 3 belonging to category A and LBP features 4-LBP features 6 corresponding to image retrieval samples 4-image retrieval samples 6 belonging to category B. The inter-class similarities between LBP feature 1 and LBP feature 4-LBP feature 6 are 87%, 67%, and 70%, respectively; the inter-class similarities between LBP feature 2 and LBP feature 4-LBP feature 6 are 91%, 75%, and 85%, respectively; and the inter-class similarities between LBP feature 3 and LBP feature 4-LBP feature 6 are 65%, 79%, and 84%, respectively.

[0198] S802: Determine a method weight corresponding to the preset feature extraction method based on the intra-class similarity and the inter-class similarity, so that the intra-class similarity is positively correlated with the method weight and the inter-class similarity is negatively correlated with the method weight.

[0199] Because the target image features extracted using different preset feature extraction methods correspond to different intra-class and inter-class similarities, if the intra-class similarity is high and the inter-class similarity is low, it indicates that the similarity between image retrieval samples of the same class is high, while the similarity between image retrieval samples of different classes is low. This effectively reflects the intra-class consistency of image retrieval samples of the same class and clarifies the inter-class differences between image retrieval samples of different classes.

[0200] Therefore, in order to intuitively reflect that the target image features corresponding to each preset feature extraction method can reflect the intra-class consistency of image retrieval samples of the same type and the inter-class differences of image retrieval samples of different types, the electronic device can determine the method weight corresponding to the preset feature extraction method for each preset feature extraction method based on the intra-class similarity and the inter-class similarity, so that the intra-class similarity is positively correlated with the method weight, and the inter-class similarity is negatively correlated with the method weight.

[0201] In the first embodiment, for each preset feature extraction method, the electronic device can calculate the average value A1 of the intra-class similarity and the average value A2 of the inter-class similarity, and calculate the method weight W1 according to the following formula: W1=0.5*A1+0.5*(1-A2).

[0202] Continuing with the example in step S801, for the extraction methods corresponding to the HOG feature, the intra-class similarities are 86%, 92%, 89%, 91%, 93%, and 87%, and the inter-class similarities are 3%, 5%, 2%, 5%, 6%, 1%, 6%, 2%, and 3%. The average values ​​of the intra-class and inter-class similarities are calculated to be 90% and 4%, respectively, i.e., A1 = 91% and A2 = 4%. Therefore, according to the formula provided in this embodiment, the method weight W1 of the extraction method corresponding to the HOG feature can be calculated to be 0.93. For the extraction method corresponding to the LBP feature, the intra-class similarities are 23%, 15%, 36%, 12%, 14%, and 9%. The inter-class similarities are 87%, 67%, 70%, 91%, 75%, 85%, 65%, 79%, and 84%. Similarly, the method weight W1 of the preset feature extraction method corresponding to the LBP feature can be calculated to be 0.2.

[0203] In the second embodiment, the first mode weight of each preset feature extraction mode can be determined based on the first embodiment, and the first mode weight W′ of the i-th preset feature extraction mode can be determined based on the first mode weight W′ of the i-th preset feature extraction mode. 1i , calculate the weight W of the i-th preset feature extraction method according to the following formula 1i : Where n is the total number of preset feature extraction methods.

[0204] Following the example of the first embodiment above, there are two preset feature extraction methods, and n is 2. Assume that the first preset feature extraction method is the extraction method corresponding to the HOG feature, and the second preset feature extraction method is the extraction method corresponding to the LBP feature. Then W′ 11 =0.93, W′ 12 =0.2. The weights W of the extraction methods corresponding to the HOG features can be calculated based on the formulas shown in this embodiment. 11 =0.82, weight W of the extraction method corresponding to the LBP feature 12 =0.18.

[0205] The following describes a parallel solution for step S802, including step A and step B:

[0206] Step A, calculating the difference between the intra-class similarity and the inter-class similarity corresponding to the preset feature extraction method;

[0207] If the difference between the intra-class similarity and inter-class similarity corresponding to a certain preset feature extraction method is large, it means that the intra-class similarity corresponding to the preset feature extraction method is large, and the inter-class similarity is small. In other words, the target image features extracted using this preset feature extraction method can better reflect the intra-class consistency of the same type of image retrieval samples and the inter-class differences of different types of image retrieval samples.

[0208] Therefore, in order to determine whether the target image features extracted using a certain preset feature extraction method can better reflect the intra-class consistency of image retrieval samples of the same type and the inter-class differences of image retrieval samples of different types, the electronic device can calculate the difference between the intra-class similarity and the inter-class similarity corresponding to the preset feature extraction method, that is, execute the above step A.

[0209] As a first calculation method, the electronic device may calculate the difference between the intra-class similarity and the inter-class similarity corresponding to the preset feature extraction method. Specifically, the electronic device may calculate the average of multiple intra-class similarities corresponding to the preset feature extraction method, calculate the average of multiple inter-class similarities corresponding to the preset feature extraction method, and subtract the average of the multiple intra-class similarities from the average of the multiple inter-class similarities to obtain the difference between the intra-class similarity and the inter-class similarity.

[0210] For example, assume that category A includes image retrieval samples A1 and A2, and category B includes image retrieval samples B1 and B2. The above image retrieval samples are subjected to feature extraction by the preset feature extraction method T1 to obtain the target image feature T 1A1 、T 1A2 、T 1B1 and T 1B2 Next, the electronic device can calculate T 1A1 With T 1A2 The similarity between 1B1 With T 1B2 The similarity between them is used to obtain the intra-class similarity L 11 and L 12 . Electronic devices can calculate T 1A1 With T 1B1 The similarity between 1A1 With T 1B2 The similarity between 1A2 With T 1B1 The similarity between 1A2 With T 1B2 The similarity between them is the inter-class similarity S 11 、S 12 、S 13 and S 14 .

[0211] Next, the electronic device can calculate L 11 With L 12 The average value of the class similarity L1 is obtained, and S is calculated. 11 、S 12 、S 13 and S 14 The electronic device can calculate L1-S1 to obtain the difference between the intra-class similarity and the inter-class similarity corresponding to the preset feature extraction method T1.

[0212] As a second calculation method, the electronic device may calculate a ratio between the intra-class similarity and the inter-class similarity corresponding to the preset feature extraction method. Specifically, the electronic device may calculate an average of multiple intra-class similarities corresponding to the preset feature extraction method, calculate an average of multiple inter-class similarities corresponding to the preset feature extraction method, and divide the average of the multiple intra-class similarities by the average of the multiple inter-class similarities to obtain a ratio between the intra-class similarity and the inter-class similarity.

[0213] As in the previous example, after calculating the intra-class similarity L1 and the inter-class similarity S1, the electronic device can calculate L1 / S1 to obtain the ratio between the intra-class similarity and the inter-class similarity corresponding to the preset feature extraction method T1.

[0214] Step B: Based on the difference, determine the method weight corresponding to the preset feature extraction method, so that the difference is positively correlated with the method weight.

[0215] If the difference corresponding to the preset feature extraction method is large, it means that the intra-class similarity corresponding to the preset feature extraction method is large, and the inter-class similarity is small. In other words, the target image features extracted using the preset feature extraction method can better reflect the intra-class consistency of image retrieval samples of the same class and the inter-class differences of image retrieval samples of different classes. Correspondingly, if the difference corresponding to the preset feature extraction method is small, it means that the intra-class similarity corresponding to the preset feature extraction method is small, and the inter-class similarity is large. In other words, the target image features extracted using the preset feature extraction method cannot better reflect the intra-class consistency of image retrieval samples of the same class and the inter-class differences of image retrieval samples of different classes.

[0216] Therefore, the difference corresponding to each preset feature extraction method can be made positively correlated with the method weight corresponding to the preset feature extraction method. Based on the difference corresponding to each preset feature extraction method, the method weight corresponding to the preset feature extraction method is determined, that is, the above step B is executed.

[0217] In one embodiment, the electronic device may map the difference corresponding to each preset feature extraction method to a value between 0 and 1 as a method weight corresponding to each preset feature extraction method. Specifically, the electronic device may calculate the sum of the differences corresponding to each preset feature extraction method. For each preset feature extraction method, the ratio of the difference corresponding to the preset feature extraction method to the sum of the differences is calculated to obtain the method weight corresponding to the preset feature extraction method.

[0218] For example, assuming that the differences corresponding to the preset feature extraction methods T1, T2, and T3 are U1, U2, and U3 respectively, then the sum of U1, U2, and U3 can be calculated to obtain U. Next, the method weights corresponding to the preset feature extraction methods T1, T2, and T3 can be calculated as: U1 / U, U2 / U, and U3 / U respectively.

[0219] As can be seen, in embodiments of the present application, the electronic device can calculate, for each preset feature extraction method, the intra-class similarity between target image features corresponding to image retrieval samples of the same category, and the inter-class similarity between target image features corresponding to image retrieval samples of different categories, where the target image features are image features extracted from the image retrieval samples using the preset feature extraction method. Based on the intra-class similarity and inter-class similarity, a method weight corresponding to the preset feature extraction method is determined, such that the intra-class similarity is positively correlated with the method weight, and the inter-class similarity is negatively correlated with the method weight. Because the target image features extracted using different preset feature extraction methods have different intra-class and inter-class similarities, if the intra-class similarity is high and the inter-class similarity is low, it indicates that the similarity between image retrieval samples of the same category is high, while the similarity between image retrieval samples of different categories is low. Furthermore, the electronic device can calculate the method weight based on the intra-class and inter-class similarities. In this way, the method weight can intuitively reflect how the target image features corresponding to each preset feature extraction method reflect the intra-class consistency of image retrieval samples of the same category and the inter-class differences of image retrieval samples of different categories.

[0220] As an implementation of an embodiment of the present application, as shown in FIG9 , after the step of determining the method weight corresponding to the preset feature extraction method based on the intra-class similarity and the inter-class similarity, the method may further include:

[0221] S901, obtaining the image to be retrieved;

[0222] The electronic device can obtain an image to be retrieved, where the image to be retrieved is an image that requires image retrieval.

[0223] S902, extracting multiple image features to be retrieved from the image to be retrieved respectively according to the multiple feature extraction methods;

[0224] Because different feature extraction methods have different degrees of importance for reflecting intra-class consistency and inter-class differences, in order to more accurately determine the retrieval results for the image to be retrieved, the electronic device can use multiple feature extraction methods to extract multiple image features of the image to be retrieved. For example, the electronic device can extract HOG features, grayscale histogram features, and SIFT features of the image to be retrieved.

[0225] S903, calculating the similarity between each image feature to be retrieved and the image feature to be compared of each image retrieval sample;

[0226] If the similarity between the image features to be retrieved and the image features to be compared of a specific image retrieval sample is high, it indicates that the image to be retrieved belongs to the category corresponding to the image retrieval sample. Therefore, the electronic device can calculate the similarity between each image feature to be retrieved and the image features to be compared of each image retrieval sample. Because the similarity between image features of the same type accurately reflects the degree of similarity between images, the feature extraction method for the image features to be compared is the same as that for the image features to be retrieved.

[0227] For example, assuming that the HOG features, grayscale histogram features, and SIFT features of image retrieval samples 1 to 6 are HOG features 1 to HOG features 6, grayscale histogram features 1 to grayscale histogram features 6, and SIFT features 1 to SIFT features 6, respectively. Then, the electronic device can calculate the similarities between the HOG features of the image to be retrieved and HOG features 1 to HOG features 6, the similarities between the grayscale histogram features of the image to be retrieved and grayscale histogram features 1 to grayscale histogram features 6, and the similarities between the SIFT features of the image to be retrieved and SIFT features 1 to SIFT features 6.

[0228] S904: Determine the image retrieval result based on the similarity and method weight corresponding to each image feature to be retrieved.

[0229] Because the method weight is positively correlated with intra-class similarity and negatively correlated with inter-class similarity, it can reflect the reference value of different feature extraction methods for image retrieval. To more accurately determine the retrieval results for the searched image, the electronic device can determine the image retrieval results based on the similarity corresponding to each image feature to be retrieved and the method weight corresponding to the feature extraction method for the image feature to be retrieved corresponding to the similarity.

[0230] It can be seen that in the embodiment of the present application, the electronic device can obtain an image to be retrieved; extract multiple image features to be retrieved of the image to be retrieved according to multiple feature extraction methods; calculate the similarity between each image feature to be retrieved and the image feature to be compared of each image retrieval sample, wherein the feature extraction method of the image feature to be compared is the same as the feature extraction method of the image feature to be retrieved; and determine the image retrieval result based on the similarity corresponding to each image feature to be retrieved and the method weight. Since the method weight represents the importance of the corresponding image feature in determining the image retrieval result, the image retrieval result can be determined based on the similarity corresponding to each image feature to be retrieved and the method weight, so that the image retrieval result of the image to be retrieved can be determined quickly and accurately.

[0231] As an implementation of an embodiment of the present application, as shown in FIG10 , the step of determining the image retrieval result based on the similarity and mode weight corresponding to each image feature to be retrieved may include:

[0232] S1001, for each image retrieval sample, performing weighted summation on each similarity corresponding to the image retrieval sample according to a corresponding weight to obtain a weighted summation result;

[0233] Because the method weights reflect the reference value of different feature extraction methods for image retrieval, the electronic device can, for each image retrieval sample, perform a weighted summation of each similarity corresponding to the image retrieval sample according to the corresponding method weights to obtain a weighted summation result. The method weight corresponding to each similarity is the weight corresponding to the feature extraction method of the image feature to be retrieved corresponding to the similarity.

[0234] For example, image retrieval samples 1-3 belong to category A, and image retrieval samples 4-5 belong to category B. The HOG features, grayscale histogram features, and SIFT features of image retrieval samples 1-5 are HOG features 1-HOG features 5, grayscale histogram features 1-grayscale histogram features 5, and SIFT features 1-SIFT features 5, respectively. The similarities between the HOG features of the image to be retrieved and HOG features 1-HOG features 5 are 89%, 92%, 87%, 12%, and 9%, respectively; the similarities between the grayscale histogram features of the image to be retrieved and grayscale histogram features 1-grayscale histogram features 5 are 45%, 43%, 53%, 23%, and 19%, respectively; and the similarities between the SIFT features of the image to be retrieved and SIFT features 1-SIFT features 5 are 32%, 28%, 34%, 31%, and 35%, respectively.

[0235] Assuming the weight for the HOG feature extraction method is 0.7, the weight for the grayscale histogram feature extraction method is 0.2, and the weight for the SIFT feature extraction method is 0.1, the weighted sum of the image to be retrieved and image retrieval sample 1 can be calculated as 89% * 0.7 + 45% * 0.2 + 32% * 0.1 = 74.5%. Similarly, the weighted sums for the image to be retrieved and image retrieval samples 2 - 5 can be calculated as 75.8%, 74.9%, 16.1%, and 13.6%, respectively.

[0236] S1002: Determine the category to which the image retrieval sample with the highest corresponding weighted result belongs as the image retrieval result corresponding to the image to be retrieved.

[0237] If the weighted summation result is high, it indicates that the image to be retrieved and the image retrieval sample corresponding to the weighted summation result have a high degree of similarity. Therefore, it can be determined that the image to be retrieved and the image retrieval sample belong to the same category. Therefore, the electronic device can determine the category of the image retrieval sample with the highest corresponding weighted summation result as the image retrieval result corresponding to the image to be retrieved.

[0238] Continuing with the example in step S1001, since 75.8% is the highest weighted sum result, image retrieval sample 2 can be determined as the sample closest to the image to be retrieved, and category A to which image retrieval sample 2 belongs can be determined as the category of the image to be retrieved.

[0239] In another embodiment, the electronic device can also pre-set the corresponding relationship between the method weight and the proportional coefficient based on the order of the method weights, multiply the similarity corresponding to each image feature to be retrieved by the proportional coefficient and sum them, and determine the category to which the image retrieval sample with the highest corresponding calculation result belongs as the category to which the image to be retrieved belongs. Continuing with the example in step S1001, assuming that the method weight of the extraction method corresponding to the HOG feature is 0.7, the method weight of the extraction method corresponding to the grayscale histogram feature is 0.2, and the method weight of the extraction method corresponding to the SIFT feature is 0.1, then the proportional coefficient corresponding to the highest method weight 0.7 can be pre-set to 0.9, the proportional coefficient corresponding to the second highest method weight 0.2 can be pre-set to 0.5, and the proportional coefficient corresponding to the smallest method weight 0.1 can be pre-set to 0.1. The calculation result corresponding to the image to be retrieved and image retrieval sample 1 is 89%*0.9+45%*0.5+32%*0.1=1.058. Similarly, the calculated results for the image to be retrieved and image retrieval samples 2-5 are 1.071, 1.082, 0.254, and 0.211, respectively. Since 1.082 is the highest calculated result, image retrieval sample 3 can be determined as the sample closest to the image to be retrieved, and category A to which image retrieval sample 3 belongs is determined as the category of the image to be retrieved.

[0240] It can be seen that in the embodiment of the present application, the electronic device can, for each image retrieval sample, perform weighted summation on each similarity corresponding to the image retrieval sample according to the corresponding dimension weight to obtain a weighted summation result, wherein the dimension weight corresponding to each similarity is the weight corresponding to the dimension of the image feature to be retrieved corresponding to the similarity; the category to which the image retrieval sample with the highest corresponding weighted result belongs is determined as the image retrieval result corresponding to the image to be retrieved. If the weighted summation result is high, it means that the similarity between the image to be retrieved and the image retrieval sample corresponding to the weighted summation result is high. Therefore, the electronic device can determine the category to which the image retrieval sample with the highest corresponding weighted summation result belongs as the image retrieval result corresponding to the image to be retrieved. In this way, image retrieval can be performed quickly and accurately. In this embodiment, the image retrieval result corresponding to the image to be retrieved is determined by weighted summation. Since the influence of each preset feature extraction method on the inter-class similarity and the intra-class similarity is comprehensively considered, the image retrieval result is more robust.

[0241] As an implementation of an embodiment of the present application, the above-mentioned image features may be a matrix. In this case, as shown in FIG11 , after the step of storing the image features of each image retrieval sample and the corresponding category label, the above-mentioned method may further include:

[0242] S1101, for each image feature, calculating the intra-class similarity between the feature values ​​of the target dimension in the image feature corresponding to image retrieval samples of the same class, and the inter-class similarity between the feature values ​​of the target dimension in the image feature corresponding to image retrieval samples of different classes;

[0243] When image features are matrices, the similarities calculated based on eigenvalues ​​of different dimensions may be different. If the eigenvalue of a certain dimension corresponds to a high intra-class similarity and a low inter-class similarity, it means that the eigenvalue of that dimension can better reflect the intra-class consistency of image retrieval samples of the same class and the inter-class differences of image retrieval samples of different classes.

[0244] Therefore, to determine the intra-class consistency and inter-class difference reflected by the feature values ​​of each dimension, the electronic device can calculate, for each image feature, the intra-class similarity between the feature values ​​of the target dimension in the image feature corresponding to image retrieval samples of the same category, and the inter-class similarity between the feature values ​​of the target dimension in the image feature corresponding to image retrieval samples of different categories. The target dimension is each row or column in the image feature.

[0245] If multiple dimensions of image features are arranged in rows, then for each image feature, the intra-class similarity between the feature values ​​of the target rows in the image feature corresponding to the image retrieval samples of the same category, and the inter-class similarity between the feature values ​​of the target rows in the image feature corresponding to the image retrieval samples of different categories can be calculated; if multiple dimensions of image features are arranged in columns, then for each image feature, the intra-class similarity between the feature values ​​of the target columns in the image feature corresponding to the image retrieval samples of the same category, and the inter-class similarity between the feature values ​​of the target columns in the image feature corresponding to the image retrieval samples of different categories can be calculated.

[0246] For example, image retrieval sample 1-image retrieval sample 2 belong to category A, and image retrieval sample 3-image retrieval sample 4 belong to category B. The HOG features of image retrieval sample 1-image retrieval sample 4 are HOG feature 1-HOG feature 4 respectively. Assuming that the HOG feature is a 3×3 matrix, the first row to the third row of the matrix are the first dimension to the third dimension respectively. Then, in HOG feature 1-HOG feature 4, the eigenvalues ​​of the first dimension are eigenvalue 11, eigenvalue 12, eigenvalue 13, and eigenvalue 14 respectively; the eigenvalues ​​of the second dimension are eigenvalue 21, eigenvalue 22, eigenvalue 23, and eigenvalue 24 respectively; the eigenvalues ​​of the third dimension are eigenvalue 31, eigenvalue 32, eigenvalue 33, and eigenvalue 34 respectively. The following takes the first dimension and the second dimension as an example to calculate the intra-class similarity and inter-class similarity:

[0247] For the first dimension, the electronic device can calculate the intra-class similarity between eigenvalue 11 and eigenvalue 12, and between eigenvalue 13 and eigenvalue 14, respectively, and obtain 34% and 23%, and calculate the inter-class similarity between eigenvalue 11 and eigenvalue 13, between eigenvalue 11 and eigenvalue 14, between eigenvalue 12 and eigenvalue 13, and between eigenvalue 12 and eigenvalue 14, respectively, and obtain 16%, 19%, 15% and 18%.

[0248] Similarly, for the second dimension, the intra-class similarities between eigenvalue 21 and eigenvalue 22, and between eigenvalue 23 and eigenvalue 24 were calculated to be 95% and 89%, and the inter-class similarities between eigenvalue 21 and eigenvalue 23, between eigenvalue 21 and eigenvalue 24, between eigenvalue 22 and eigenvalue 23, and between eigenvalue 22 and eigenvalue 24 were calculated to be 4%, 5%, 3%, and 2%.

[0249] In one embodiment, the image feature may be a multi-dimensional vector. In this case, the target dimension may be one dimension in the multi-dimensional vector.

[0250] S1102: Determine the dimension weight corresponding to the target dimension based on the intra-class similarity and the inter-class similarity, so that the intra-class similarity is positively correlated with the dimension weight, and the inter-class similarity is negatively correlated with the dimension weight.

[0251] Due to the different eigenvalues ​​of different dimensions, the corresponding intra-class similarity and inter-class similarity are different. If the intra-class similarity is high and the inter-class similarity is low, it means that the similarity between image retrieval samples of the same class is high, while the similarity between image retrieval samples of different classes is low. This can better reflect the intra-class consistency of image retrieval samples of the same class and clarify the inter-class differences of image retrieval samples of different classes.

[0252] Therefore, in order to intuitively demonstrate that the eigenvalue of each dimension can reflect the intra-class consistency of image retrieval samples of the same class and the inter-class difference of image retrieval samples of different classes, the electronic device can determine the dimension weight corresponding to the target dimension based on the intra-class similarity and the inter-class similarity, so that the intra-class similarity is positively correlated with the dimension weight, and the inter-class similarity is negatively correlated with the dimension weight.

[0253] In the first embodiment, for the eigenvalues ​​of each dimension in the image features, the electronic device can calculate the average intra-class similarity A3 and the average inter-class similarity A4 between the eigenvalues ​​of the dimension, and calculate the dimension weight W2 according to the following formula: W2 = 0.5*A3 + 0.5*(1-A4).

[0254] Continuing with the example in step S901, for the first dimension, the intra-class similarity is 34% and 23%, and the inter-class similarity is 16%, 19%, 15%, and 18%. The average values ​​of the intra-class similarity and the inter-class similarity are calculated to be 28.5% and 17%, respectively, that is, A3 = 28.5% and A4 = 17%. Then, according to the formula provided in this embodiment, the dimension weight W2 = 0.56 of the first dimension can be calculated. Similarly, for the second dimension, the intra-class similarity is 95% and 89%, and the inter-class similarity is 4%, 5%, 3%, and 2%. It can be calculated that the dimension weight W2 of the second dimension is 0.94.

[0255] In the second embodiment, the first dimension weight of each dimension can be determined based on the first embodiment, and the first dimension weight W2′2 of the i-th dimension can be determined based on the first dimension weight W2′2 of the i-th dimension. i , calculate the dimension weight W of the i-th dimension according to the following formula 2i : Where n is the total number of dimensions of image features.

[0256] Following the example in the first embodiment above, assuming that the dimension weights corresponding to the first dimension to the third dimension of the HOG feature are 0.56, 0.94, and 0.30 respectively, then W′ 21 =0.56, W′ 22 =0.94, W′ 23 =0.30. The weights W of the first dimension can be calculated based on the formula shown in this embodiment. 21 =0.31, the second dimension weight W 22 =0.52, the third dimension weight W 23 =0.17.

[0257] As can be seen, in embodiments of the present application, the electronic device can calculate, for each image feature, the intra-class similarity between feature values ​​of a target dimension in the image feature corresponding to image retrieval samples of the same category, and the inter-class similarity between feature values ​​of the target dimension in the image feature corresponding to image retrieval samples of different categories, where the target dimension is each row or column of the image feature. Based on the intra-class similarity and inter-class similarity, a dimension weight corresponding to the target dimension is determined, such that the intra-class similarity is positively correlated with the dimension weight, and the inter-class similarity is negatively correlated with the dimension weight. Because feature values ​​of different dimensions in the image feature have different corresponding intra-class and inter-class similarities, if the intra-class similarity is high and the inter-class similarity is low, it indicates that the similarity between image retrieval samples of the same category is high, while the similarity between image retrieval samples of different categories is low. Furthermore, the electronic device can calculate the dimension weight based on the intra-class and inter-class similarities. In this way, the dimension weight can intuitively reflect how the feature values ​​corresponding to each dimension reflect the intra-class consistency of image retrieval samples of the same category and the inter-class differences of image retrieval samples of different categories.

[0258] As an implementation of an embodiment of the present application, as shown in FIG12 , after the step of determining the dimension weight corresponding to the target dimension based on the intra-class similarity and the inter-class similarity, the method may further include:

[0259] S1201, obtaining the image to be retrieved;

[0260] The electronic device can obtain an image to be retrieved, where the image to be retrieved is an image that requires image retrieval.

[0261] S1202, extracting the image features to be retrieved of the image to be retrieved according to the preset feature extraction method;

[0262] In order to determine the features of the image to be retrieved and then perform image retrieval on the image to be retrieved, the electronic device can extract the image features to be retrieved of the image to be retrieved according to a preset feature extraction method.

[0263] S1203, calculating the similarity between the feature value of each dimension of the image feature to be retrieved and the feature value of the corresponding dimension of the image feature to be compared of each image retrieval sample;

[0264] If the similarity between the feature value of a certain dimension of the image feature to be retrieved and the feature value of the corresponding dimension of a certain image retrieval sample is high, it means that the image to be retrieved may belong to the category corresponding to the image retrieval sample. Therefore, the electronic device can calculate the similarity between the feature value of each dimension of the image feature to be retrieved and the feature value of the corresponding dimension of the image feature to be compared of each image retrieval sample. Since the similarity between the same type of image features can accurately reflect the degree of similarity between images, the feature extraction method of the image feature to be compared is the same as the feature extraction method of the image feature to be retrieved;

[0265] For example, suppose the HOG feature is a 3×3 matrix, with the first to third rows of the matrix being the first dimension and the third dimension, respectively. The HOG features of image retrieval sample 1 and image retrieval sample 3 are HOG features 1 and HOG features 3, respectively. The eigenvalues ​​of the first, second, and third dimensions of HOG feature 1 are 11, 12, and 13, respectively. The eigenvalues ​​of the first, second, and third dimensions of HOG feature 2 are 21, 22, and 23, respectively. The eigenvalues ​​of the first, second, and third dimensions of HOG feature 3 are 31, 32, and 33, respectively. The eigenvalues ​​of the first, second, and third dimensions of the HOG feature of the image to be retrieved are 41, 42, and 43, respectively. Then, the electronic device can calculate the similarity between eigenvalue 41 and eigenvalue 11, eigenvalue 21, and eigenvalue 31 respectively, can calculate the similarity between eigenvalue 42 and eigenvalue 12, eigenvalue 22, and eigenvalue 32 respectively, and can calculate the similarity between eigenvalue 43 and eigenvalue 13, eigenvalue 23, and eigenvalue 33 respectively.

[0266] S1204: Determine an image retrieval result based on the similarity corresponding to the feature value of each dimension in the image feature to be retrieved and the dimension weight.

[0267] Since dimension weights are positively correlated with intra-class similarity and negatively correlated with inter-class similarity, dimension weights can reflect the reference value of different dimensions in image features for image retrieval. In order to more accurately determine the retrieval results of the retrieved image, the electronic device can determine the image retrieval results based on the similarity corresponding to the feature value of each dimension in the image features to be retrieved, and the dimension weight corresponding to the similarity is the dimension weight corresponding to the dimension of the image features to be retrieved corresponding to the similarity.

[0268] It can be seen that in the embodiment of the present application, the electronic device can obtain the image to be retrieved; extract the image features to be retrieved of the image to be retrieved according to a preset feature extraction method; calculate the similarity between the feature value of each dimension of the image features to be retrieved and the feature value of the corresponding dimension of the image features to be compared of each image retrieval sample, wherein the feature extraction method of the image features to be compared is the same as the feature extraction method of the image features to be retrieved; and determine the image retrieval result based on the similarity corresponding to the feature value of each dimension in the image features to be retrieved and the dimension weight. Since the dimension weight represents the importance of the feature value of the corresponding dimension in determining the image retrieval result, the image retrieval result is determined based on the similarity corresponding to the feature value of each dimension in the image features to be retrieved and the dimension weight, so that the image retrieval result of the image to be retrieved can be determined quickly and accurately.

[0269] As an implementation of an embodiment of the present application, as shown in FIG13 , the step of determining the image retrieval result based on the similarity corresponding to the feature value of each dimension in the image feature to be retrieved and the dimension weight may include:

[0270] S1301, for each image retrieval sample, performing weighted summation on each similarity corresponding to the image retrieval sample according to the corresponding dimension weight to obtain a weighted summation result;

[0271] Because dimension weights reflect the reference value of different dimensions of image features for image retrieval, the electronic device can, for each image retrieval sample, perform a weighted summation of each similarity corresponding to the image retrieval sample according to the corresponding dimension weight to obtain a weighted summation result. The dimension weight corresponding to each similarity is the weight corresponding to the dimension of the image feature to be retrieved corresponding to the similarity.

[0272] For example, assume that the dimension weights corresponding to the first, second, and third dimensions in the HOG feature are 0.5, 0.2, and 0.3, respectively. Image retrieval sample 1-image retrieval sample 2 belong to category A, and image retrieval sample 3-image retrieval sample 4 belong to category B. The HOG features corresponding to image retrieval sample 1-image retrieval sample 4 are HOG feature 1-HOG feature 4, respectively. The similarities corresponding to the eigenvalues ​​of the first dimension in HOG feature 1-HOG feature 4 are 76%, 65%, 11%, and 6%, respectively. The similarities corresponding to the eigenvalues ​​of the second dimension in HOG feature 1-HOG feature 4 are 56%, 45%, 19%, and 16%, respectively. The similarities corresponding to the eigenvalues ​​of the third dimension in HOG feature 1-HOG feature 4 are 72%, 52%, 11%, and 14%, respectively.

[0273] Then, we can use the formula 76%*0.5+56%*0.2+72%*0.3=0.71 to calculate the calculation result between the image to be retrieved and image retrieval sample 1. Similarly, we can calculate that the calculation results corresponding to the image to be retrieved and image retrieval samples 2-image retrieval samples 4 are 0.57, 0.13, and 0.10 respectively.

[0274] S1302: Determine the category to which the image retrieval sample with the highest corresponding weighted result belongs as the image retrieval result corresponding to the image to be retrieved.

[0275] If the weighted summation result is high, it indicates that the image to be retrieved and the image retrieval sample corresponding to the weighted summation result have a high degree of similarity. Therefore, it can be determined that the image to be retrieved and the image retrieval sample belong to the same category. Therefore, the electronic device can determine the category of the image retrieval sample with the highest corresponding weighted summation result as the image retrieval result corresponding to the image to be retrieved.

[0276] Continuing with the example in step S1301, since 0.71 is the highest calculated result, the image retrieval sample 1 can be determined as the sample closest to the image to be retrieved, and the category A to which the image retrieval sample 1 belongs can be determined as the category of the image to be retrieved.

[0277] In another embodiment, the electronic device can also pre-set the correspondence between dimension weights and proportional coefficients based on the order of the method weights, multiply the similarity corresponding to each dimension by the proportional coefficient, and sum them. The category to which the image retrieval sample with the highest corresponding calculation result belongs is determined as the category to which the image to be retrieved belongs. Continuing with the example in step S1301, the proportional coefficient corresponding to the highest dimension weight is pre-set to 0.9, the proportional coefficient corresponding to the second highest dimension weight is pre-set to 0.5, and the proportional coefficient corresponding to the smallest dimension weight is pre-set to 0.1. The calculated result corresponding to the image to be retrieved and image retrieval sample 1 is calculated to be 76%*0.9+56%*0.5+72%*0.1=1.036. Similarly, the calculated results corresponding to the image to be retrieved and image retrieval samples 2-image retrieval samples 4 can be calculated to be 0.862, 0.205, and 0.148, respectively. Since 1.036 is the highest calculated result, image retrieval sample 1 can be determined as the sample closest to the image to be retrieved, and category A to which image retrieval sample 1 belongs is determined as the category of the image to be retrieved.

[0278] It can be seen that in the embodiment of the present application, the electronic device can, for each image retrieval sample, perform weighted summation on each similarity corresponding to the image retrieval sample according to the corresponding dimension weight to obtain a weighted summation result, wherein the dimension weight corresponding to each similarity is the weight corresponding to the dimension of the image feature to be retrieved corresponding to the similarity; the category to which the image retrieval sample with the highest corresponding weighted result belongs is determined as the image retrieval result corresponding to the image to be retrieved. If the weighted summation result is high, it means that the similarity between the image to be retrieved and the image retrieval sample corresponding to the weighted summation result is high. Therefore, the electronic device can determine the category to which the image retrieval sample with the highest corresponding weighted summation result belongs as the image retrieval result corresponding to the image to be retrieved. In this way, image retrieval can be performed quickly and accurately.

[0279] As an implementation of an embodiment of the present application, after the step of storing the image features and corresponding category labels of each image retrieval sample, the method may further include:

[0280] For each preset feature extraction method, the intra-class similarity between the target image features corresponding to image retrieval samples of the same category is calculated, and the inter-class similarity between the target image features corresponding to image retrieval samples of different categories is calculated. For each image feature, the third similarity between the feature values ​​of the target dimension in the image features corresponding to the image retrieval samples of the same category is calculated, and the fourth similarity between the feature values ​​of the target dimension in the image features corresponding to the image retrieval samples of different categories is calculated. Based on the intra-class similarity and the inter-class similarity, the method weight corresponding to the preset feature extraction method is determined. And based on the third similarity and the fourth similarity, the dimension weight corresponding to the target dimension is determined.

[0281] Next, the electronic device can obtain an image to be retrieved; extract multiple image features to be retrieved from the image to be retrieved according to multiple feature extraction methods; calculate the fifth similarity between each image feature to be retrieved and the image feature to be compared of each image retrieval sample, and the sixth similarity between the feature value of each dimension of the image feature to be retrieved and the feature value of the corresponding dimension of the image feature to be compared of each image retrieval sample; and determine the image retrieval result based on the fifth similarity and method weight corresponding to each image feature to be retrieved, and the sixth similarity and dimension weight corresponding to the feature value of each dimension of the image feature to be retrieved. In this way, the accuracy of image retrieval can be further improved.

[0282] As an implementation method of an embodiment of the present application, a schematic diagram of an image retrieval sample registration method can be shown in FIG14 , where an electronic device can obtain an original image sample and a category label corresponding to the original image sample. Image transformation processing 1, image transformation processing 2…image transformation processing n are performed on the original image sample, respectively. That is, n different transformation processes are performed on the original image sample. Then, transformed image 1, transformed image 2…transformed image n are obtained, and feature extraction is performed on the obtained n transformed images and the original image sample to obtain image feature 0 corresponding to the original image sample, and image feature 1, image feature 2…image feature n corresponding to transformed images 1-transformed images n, respectively.

[0283] In this way, the electronic device can register image features 0-n and their corresponding category labels and store them in the category database. If the number of original image samples is m, and n different transformations are performed on each original image sample, a total of m*(n+1) image features can be obtained.

[0284] Furthermore, the electronic device can perform feature selection, that is, it can determine the intra-class consistency and inter-class difference that can be reflected by the image features corresponding to each image transformation process. Specifically, the electronic device can calculate the intra-class similarity between the target image features corresponding to images of the same category in the category database for each image transformation process, and calculate the inter-class similarity between the target image features corresponding to images of different categories, wherein the target image features are image features extracted from the image retrieval sample using this image transformation process. Then, based on the intra-class similarity and the inter-class similarity, the method weight corresponding to the image transformation process is determined. Then, the correspondence between each image transformation process and the method weight is recorded in the feature weight table.

[0285] As can be seen, in the embodiments of this application, data augmentation technology is used to generate a set of registered image samples that differ from the original image samples through various data augmentation methods, thereby more comprehensively describing the diversity of possible intra-class differences among the registered image samples. By determining the most suitable feature type combination for the current task through feature selection, the accuracy of image retrieval based on the enhanced database can be improved, and the adaptability and usability of the image retrieval algorithm for small sample task scenarios can be enhanced.

[0286] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0287] Corresponding to the above-mentioned method for registering an image retrieval sample, an embodiment of the present application further provides a device for registering an image retrieval sample. The following introduces the device for registering an image retrieval sample provided in an embodiment of the present application.

[0288] As shown in FIG15 , a device for registering image retrieval samples includes:

[0289] The image acquisition module 1501 is used to acquire original image samples and category labels corresponding to the original image samples;

[0290] The transformation processing module 1502 is used to perform different transformation processes on the original image samples to obtain multiple transformed images;

[0291] A feature extraction module 1503 is configured to extract image features of each image retrieval sample according to a preset feature extraction method, wherein the image retrieval sample includes the original image sample and the multiple transformed images, and any two images included in the image retrieval sample are not completely identical;

[0292] The feature storage module 1504 is used to store the image features and corresponding category labels of each image retrieval sample for use in image retrieval.

[0293] It can be seen that in the solution provided by the embodiment of the present application, the electronic device can obtain the original image sample and the category label corresponding to the original image sample; perform different transformation processing on the original image sample to obtain multiple transformed images; extract the image features of each image retrieval sample according to the preset feature extraction method, wherein the image retrieval sample includes the original image sample and multiple transformed images, and any two images included in the image retrieval sample are not exactly the same; store the image features of each image retrieval sample and the corresponding category label for image retrieval. Since the transformed image is obtained by transforming the original image sample, and the transformation processing does not change the category to which the image belongs, the transformed image and the original image sample can be used as image retrieval samples of the same category. Even if the number of original image samples is limited, the original image sample can be transformed to obtain multiple transformed images, which can make the image retrieval sample more completely reflect the feature diversity of samples of the same category and clarify the key difference features between samples of different categories. In this way, the number of image retrieval samples can be expanded and the accuracy of image retrieval can be improved.

[0294] As an implementation of an embodiment of the present application, the transformation processing module 1502 may include:

[0295] The transformation processing submodule is used to transform each original image sample using at least one of the following preset processing methods to obtain multiple transformed images, wherein the preset processing methods include: region of interest perturbation processing, brightness transformation processing, and pixel transformation processing.

[0296] As an implementation of an embodiment of the present application, the above-mentioned transformation processing submodule may include:

[0297] a position determining unit, configured to determine, for each original image sample, a position of a region of interest of the original image sample;

[0298] a position changing unit, configured to move at least one side of the position of the region of interest by a target distance along a target direction, and use the intersection of a straight line on which the moved side is located and a straight line on which its adjacent side is located as a vertex of the region of interest, to obtain a changed position of the region of interest, wherein the target direction is the direction of a straight line on which the adjacent side of the side to be moved is located, and the ratio of the target distance to the length of the adjacent side of the side to be moved is a target ratio;

[0299] A region rotation unit, configured to rotate the position of the region of interest by a target angle to obtain a changed position of the region of interest;

[0300] The image cropping unit is used to crop the original image sample according to the changed position of the region of interest to obtain a transformed image.

[0301] As an implementation of an embodiment of the present application, the above-mentioned position determination unit may include:

[0302] a first position determining subunit, configured to, if information about the region of interest of the original image sample is obtained, calculate a minimum bounding rectangle of the region of interest according to the information about the region of interest as the position of the region of interest of the original image sample, wherein the information about the region of interest is used to identify the position of the region of interest;

[0303] The second position determining subunit is configured to determine the entire area of ​​the original image sample as the position of the region of interest if the region of interest information of the original image sample is not acquired.

[0304] As an implementation of the embodiment of the present application, the above-mentioned device may further include:

[0305] A range acquisition module, configured to acquire a preset ratio range, wherein the preset ratio range is predetermined based on a disturbance degree of a region of interest;

[0306] The ratio determination module is used to determine a corresponding target ratio for each edge from the preset ratio range.

[0307] As an implementation of an embodiment of the present application, the above-mentioned preset feature extraction method may include multiple feature extraction methods. In this case, the above-mentioned device may further include:

[0308] an intra-class similarity calculation module, configured to calculate, for each preset feature extraction method, the intra-class similarity between target image features corresponding to image retrieval samples of the same category, and to calculate the inter-class similarity between target image features corresponding to image retrieval samples of different categories, wherein the target image features are image features extracted from the image retrieval samples using the preset feature extraction method;

[0309] The first weight determination module is used to determine the method weight corresponding to the preset feature extraction method based on the intra-class similarity and the inter-class similarity, so that the intra-class similarity is positively correlated with the method weight, and the inter-class similarity is negatively correlated with the method weight; or, calculate the difference between the intra-class similarity and the inter-class similarity corresponding to the preset feature extraction method; based on the difference, determine the method weight corresponding to the preset feature extraction method, so that the difference is positively correlated with the method weight.

[0310] As an implementation of the embodiment of the present application, the above-mentioned device may further include:

[0311] A first image acquisition module, used to acquire an image to be retrieved;

[0312] A first feature extraction module is used to extract multiple image features to be retrieved of the image to be retrieved according to the multiple feature extraction methods;

[0313] an inter-class similarity calculation module, configured to calculate the similarity between each image feature to be retrieved and the image feature to be compared of each image retrieval sample, wherein the feature extraction method of the image feature to be compared is the same as the feature extraction method of the image feature to be retrieved;

[0314] The first result determination module is used to determine the image retrieval result based on the similarity and method weight corresponding to each image feature to be retrieved.

[0315] As an implementation of an embodiment of the present application, the first result determination module may include:

[0316] A first calculation submodule is configured to perform weighted summation on each similarity corresponding to each image retrieval sample according to a corresponding method weight, to obtain a weighted summation result, wherein the method weight corresponding to each similarity is the weight corresponding to the feature extraction method of the image feature to be retrieved corresponding to the similarity;

[0317] The first result determination submodule is configured to determine the category to which the image retrieval sample with the highest corresponding weighted result belongs as the image retrieval result corresponding to the image to be retrieved.

[0318] As an implementation manner of the embodiment of the present application, the above-mentioned image feature may be a matrix. In this case, the above-mentioned apparatus may further include:

[0319] a third similarity calculation module, configured to calculate, for each image feature, an intra-class similarity between eigenvalues ​​of a target dimension in the image feature corresponding to image retrieval samples of the same category, and an inter-class similarity between eigenvalues ​​of the target dimension in the image feature corresponding to image retrieval samples of different categories, wherein the target dimension is each row or column of the matrix;

[0320] The second weight determination module is used to determine the dimension weight corresponding to the target dimension based on the intra-class similarity and the inter-class similarity, so that the intra-class similarity is positively correlated with the dimension weight, and the inter-class similarity is negatively correlated with the dimension weight.

[0321] As an implementation of the embodiment of the present application, the above-mentioned device may further include:

[0322] A second image acquisition module, used to acquire the image to be retrieved;

[0323] A second feature extraction module is used to extract the image features to be retrieved of the image to be retrieved according to the preset feature extraction method;

[0324] a fourth similarity calculation module, configured to calculate the similarity between the feature value of each dimension of the image feature to be retrieved and the feature value of the corresponding dimension of the image feature to be compared of each image retrieval sample, wherein the feature extraction method of the image feature to be compared is the same as the feature extraction method of the image feature to be retrieved;

[0325] The second result determination module is used to determine the image retrieval result based on the similarity corresponding to the feature value of each dimension in the image feature to be retrieved and the dimension weight.

[0326] As an implementation of the embodiment of the present application, the second result determination module may include:

[0327] The second calculation submodule is used to perform weighted summation on each similarity corresponding to each image retrieval sample according to the corresponding dimension weight to obtain a weighted summation result, wherein the dimension weight corresponding to each similarity is the weight corresponding to the dimension of the image feature to be retrieved corresponding to the similarity;

[0328] The second result determination submodule is configured to determine the category to which the image retrieval sample with the highest corresponding weighted result belongs as the image retrieval result corresponding to the image to be retrieved.

[0329] The present application also provides an electronic device, as shown in FIG16 , including:

[0330] Memory 1601, used for storing computer programs;

[0331] The processor 1602 is configured to implement the steps of the image retrieval sample registration method described in any of the above embodiments when executing the program stored in the memory 1601 .

[0332] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 1602, the communication interface, and the memory 1601 communicate with each other via the communication bus.

[0333] It can be seen that in the solution provided by the embodiment of the present application, the electronic device can obtain the original image sample and the category label corresponding to the original image sample; perform different transformation processing on the original image sample to obtain multiple transformed images; extract the image features of each image retrieval sample according to the preset feature extraction method, wherein the image retrieval sample includes the original image sample and multiple transformed images, and any two images included in the image retrieval sample are not exactly the same; store the image features of each image retrieval sample and the corresponding category label for image retrieval. Since the transformed image is obtained by transforming the original image sample, and the transformation processing does not change the category to which the image belongs, the transformed image and the original image sample can be used as image retrieval samples of the same category. Even if the number of original image samples is limited, the original image sample can be transformed to obtain multiple transformed images, which can make the image retrieval sample more completely reflect the feature diversity of samples of the same category and clarify the key difference features between samples of different categories. In this way, the number of image retrieval samples can be expanded and the accuracy of image retrieval can be improved.

[0334] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0335] The communication interface is used for communication between the above electronic device and other devices.

[0336] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0337] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0338] In another embodiment provided by the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the steps of any of the above-mentioned image retrieval sample registration methods are implemented.

[0339] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the image retrieval sample registration methods in the above embodiments.

[0340] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state drive (SSD).

[0341] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0342] Each embodiment in this specification is described in a related manner. Similar portions between embodiments can be referenced to each other. Each embodiment focuses on the differences from other embodiments. In particular, since the apparatus, system, electronic device, computer-readable storage medium, and computer program product are generally similar to the method embodiments, their descriptions are relatively simplified. For related portions, reference can be made to the descriptions of the method embodiments.

[0343] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

[0344] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A registration method for image retrieval samples, characterized in that: The method comprises: Obtaining an original image sample and a category label corresponding to the original image sample; Performing different transformation processes on the original image samples to obtain multiple transformed images; Extracting image features of each image retrieval sample according to a preset feature extraction method, wherein the image retrieval sample includes the original image sample and the multiple transformed images, and any two images included in the image retrieval sample are not completely identical; The image features and corresponding category labels of each image retrieval sample are stored for use in image retrieval.

2. The method according to claim 1, characterized in that The step of performing different transformation processes on the original image samples to obtain a plurality of transformed images includes: For each original image sample, at least one of the following preset processing methods is used to transform the original image sample to obtain multiple transformed images, wherein the preset processing methods include: region of interest disturbance processing, brightness transformation processing, and pixel transformation processing.

3. The method according to claim 2, characterized in that The method of perturbation processing in the region of interest includes: For each original image sample, determining a position of a region of interest of the original image sample; Moving at least one side of the region of interest position by a target distance along a target direction, and using the intersection of a straight line on which the moved side lies and a straight line on which its adjacent side lies as a vertex of the region of interest, to obtain a modified region of interest position, wherein the target direction is the direction of a straight line on which the adjacent side of the side to be moved lies, and the ratio of the target distance to the length of the adjacent side of the side to be moved is a target ratio; and / or rotating the region of interest position by a target angle to obtain a modified region of interest position; The original image sample is cropped according to the changed position of the region of interest to obtain a transformed image.

4. The method according to claim 3, characterized in that The step of determining the position of the region of interest of the original image sample comprises: If the region of interest information of the original image sample is obtained, the minimum bounding rectangle of the region of interest is calculated according to the region of interest information as the position of the region of interest of the original image sample, wherein the region of interest information is used to identify the position of the region of interest; or If the region of interest information of the original image sample is not obtained, the entire region of the original image sample is determined as the region of interest position.

5. The method according to claim 3, characterized in that Before the step of moving at least one side of the region of interest by a target distance along a target direction, the method further includes: Obtaining a preset ratio range, wherein the preset ratio range is predetermined based on a disturbance degree of the region of interest; A corresponding target ratio is determined for each edge from the preset ratio range.

6. The method according to any one of claims 1 to 5, characterized in that The preset feature extraction method includes multiple feature extraction methods; After the step of storing the image features and corresponding category labels of each image retrieval sample, the method further includes: For each preset feature extraction method, calculate the intra-class similarity between target image features corresponding to image retrieval samples of the same category, and calculate the inter-class similarity between target image features corresponding to image retrieval samples of different categories, wherein the target image features are image features extracted from the image retrieval samples using the preset feature extraction method; Based on the intra-class similarity and the inter-class similarity, determine the method weight corresponding to the preset feature extraction method, so that the intra-class similarity is positively correlated with the method weight, and the inter-class similarity is negatively correlated with the method weight; or, calculate the difference between the intra-class similarity and the inter-class similarity corresponding to the preset feature extraction method; based on the difference, determine the method weight corresponding to the preset feature extraction method, so that the difference is positively correlated with the method weight.

7. The method according to claim 6, characterized in that After the step of determining a method weight corresponding to the preset feature extraction method based on the intra-class similarity and the inter-class similarity, the method further includes: Get the image to be retrieved; extracting a plurality of image features to be retrieved of the image to be retrieved respectively according to the plurality of feature extraction methods; Calculating the similarity between each image feature to be retrieved and the image feature to be compared of each image retrieval sample, wherein the feature extraction method of the image feature to be compared is the same as the feature extraction method of the image feature to be retrieved; The image retrieval results are determined based on the similarity and method weight corresponding to each image feature to be retrieved.

8. The method according to claim 7, characterized in that The step of determining the image retrieval result based on the similarity and mode weight corresponding to each image feature to be retrieved includes: For each image retrieval sample, each similarity corresponding to the image retrieval sample is weighted and summed according to the corresponding method weight to obtain a weighted summation result, wherein the method weight corresponding to each similarity is the weight corresponding to the feature extraction method of the image feature to be retrieved corresponding to the similarity; The category to which the image retrieval sample with the highest corresponding weighted result belongs is determined as the image retrieval result corresponding to the image to be retrieved.

9. The method according to any one of claims 1 to 5, characterized in that The image feature is a matrix; After the step of storing the image features and corresponding category labels of each image retrieval sample, the method further includes: For each image feature, calculate the intra-class similarity between the eigenvalues ​​of the target dimension in the image feature corresponding to the image retrieval samples of the same category, and the inter-class similarity between the eigenvalues ​​of the target dimension in the image feature corresponding to the image retrieval samples of different categories, wherein the target dimension is each row or column of the matrix; Based on the intra-class similarity and the inter-class similarity, a dimension weight corresponding to the target dimension is determined, so that the intra-class similarity is positively correlated with the dimension weight, and the inter-class similarity is negatively correlated with the dimension weight.

10. The method according to claim 9, characterized in that After the step of determining the dimension weight corresponding to the target dimension based on the intra-class similarity and the inter-class similarity, the method further includes: Get the image to be retrieved; Extracting the image features to be retrieved of the image to be retrieved according to the preset feature extraction method; Calculating the similarity between the feature value of each dimension of the image feature to be retrieved and the feature value of the corresponding dimension of the image feature to be compared of each image retrieval sample, wherein the feature extraction method of the image feature to be compared is the same as the feature extraction method of the image feature to be retrieved; An image retrieval result is determined based on the similarity corresponding to the feature value of each dimension in the image features to be retrieved and the dimension weight.

11. The method according to claim 10, characterized in that The step of determining the image retrieval result based on the similarity corresponding to the feature value of each dimension in the image feature to be retrieved and the dimension weight includes: For each image retrieval sample, each similarity corresponding to the image retrieval sample is weighted and summed according to the corresponding dimension weight to obtain a weighted summation result, wherein the dimension weight corresponding to each similarity is the weight corresponding to the dimension of the image feature to be retrieved corresponding to the similarity; The category to which the image retrieval sample with the highest corresponding weighted result belongs is determined as the image retrieval result corresponding to the image to be retrieved.

12. A registration device for image retrieval samples, characterized in that: The device comprises: An image acquisition module is used to acquire original image samples and category labels corresponding to the original image samples; A transformation processing module, configured to perform different transformation processes on the original image samples to obtain a plurality of transformed images; a feature extraction module, configured to extract image features of each image retrieval sample according to a preset feature extraction method, wherein the image retrieval sample includes the original image sample and the plurality of transformed images, and any two images included in the image retrieval sample are not completely identical; The feature storage module is used to store the image features and corresponding category labels of each image retrieval sample for use in image retrieval.

13. The device according to claim 12, characterized in that The transformation processing module includes: a transformation processing submodule, configured to transform each original image sample using at least one of the following preset processing methods to obtain a plurality of transformed images, wherein the preset processing methods include: region of interest perturbation processing, brightness transformation processing, and pixel transformation processing; The transformation processing submodule includes: a position determining unit, configured to determine, for each original image sample, a position of a region of interest of the original image sample; a position changing unit, configured to move at least one side of the position of the region of interest by a target distance along a target direction, and use the intersection of a straight line on which the moved side is located and a straight line on which its adjacent side is located as a vertex of the region of interest, to obtain a changed position of the region of interest, wherein the target direction is the direction of a straight line on which the adjacent side of the side to be moved is located, and the ratio of the target distance to the length of the adjacent side of the side to be moved is a target ratio; A region rotation unit, configured to rotate the position of the region of interest by a target angle to obtain a changed position of the region of interest; An image cropping unit, configured to crop the original image sample according to the changed position of the region of interest to obtain a transformed image; The position determination unit includes: a first position determining subunit, configured to, if information about the region of interest of the original image sample is obtained, calculate a minimum bounding rectangle of the region of interest according to the information about the region of interest as the position of the region of interest of the original image sample, wherein the information about the region of interest is used to identify the position of the region of interest; A second position determining subunit is configured to determine the entire area of ​​the original image sample as the position of the region of interest if the region of interest information of the original image sample is not obtained; The device further comprises: A range acquisition module, configured to acquire a preset ratio range, wherein the preset ratio range is predetermined based on a disturbance degree of a region of interest; a ratio determination module, configured to determine a corresponding target ratio for each edge from the preset ratio range; The preset feature extraction method includes multiple feature extraction methods; The device further comprises: an intra-class similarity calculation module, configured to calculate, for each preset feature extraction method, the intra-class similarity between target image features corresponding to image retrieval samples of the same category, and to calculate the inter-class similarity between target image features corresponding to image retrieval samples of different categories, wherein the target image features are image features extracted from the image retrieval samples using the preset feature extraction method; A first weight determination module is configured to determine a method weight corresponding to the preset feature extraction method based on the intra-class similarity and the inter-class similarity, so that the intra-class similarity is positively correlated with the method weight, and the inter-class similarity is negatively correlated with the method weight; or to calculate the difference between the intra-class similarity and the inter-class similarity corresponding to the preset feature extraction method; and based on the difference, determine the method weight corresponding to the preset feature extraction method, so that the difference is positively correlated with the method weight; The device further comprises: A first image acquisition module, used to acquire an image to be retrieved; A first feature extraction module is used to extract multiple image features to be retrieved of the image to be retrieved according to the multiple feature extraction methods; an inter-class similarity calculation module, configured to calculate the similarity between each image feature to be retrieved and the image feature to be compared of each image retrieval sample, wherein the feature extraction method of the image feature to be compared is the same as the feature extraction method of the image feature to be retrieved; A first result determination module is used to determine the image retrieval result based on the similarity and method weight corresponding to each image feature to be retrieved; The first result determination module includes: A first calculation submodule is configured to perform weighted summation on each similarity corresponding to each image retrieval sample according to a corresponding method weight, to obtain a weighted summation result, wherein the method weight corresponding to each similarity is the weight corresponding to the feature extraction method of the image feature to be retrieved corresponding to the similarity; A first result determination submodule is configured to determine the category to which the image retrieval sample with the highest corresponding weighted result belongs as the image retrieval result corresponding to the image to be retrieved; The image feature is a matrix; The device further comprises: a third similarity calculation module, configured to calculate, for each image feature, an intra-class similarity between eigenvalues ​​of a target dimension in the image feature corresponding to image retrieval samples of the same category, and an inter-class similarity between eigenvalues ​​of the target dimension in the image feature corresponding to image retrieval samples of different categories, wherein the target dimension is each row or column of the matrix; A second weight determination module is configured to determine a dimension weight corresponding to the target dimension based on the intra-class similarity and the inter-class similarity, so that the intra-class similarity is positively correlated with the dimension weight, and the inter-class similarity is negatively correlated with the dimension weight; The device further comprises: A second image acquisition module, used to acquire the image to be retrieved; A second feature extraction module is used to extract the image features to be retrieved of the image to be retrieved according to the preset feature extraction method; a fourth similarity calculation module, configured to calculate the similarity between the feature value of each dimension of the image feature to be retrieved and the feature value of the corresponding dimension of the image feature to be compared of each image retrieval sample, wherein the feature extraction method of the image feature to be compared is the same as the feature extraction method of the image feature to be retrieved; A second result determination module is used to determine the image retrieval result based on the similarity corresponding to the feature value of each dimension in the image feature to be retrieved and the dimension weight; The second result determination module includes: The second calculation submodule is used to perform weighted summation on each similarity corresponding to each image retrieval sample according to the corresponding dimension weight to obtain a weighted summation result, wherein the dimension weight corresponding to each similarity is the weight corresponding to the dimension of the image feature to be retrieved corresponding to the similarity; The second result determination submodule is configured to determine the category to which the image retrieval sample with the highest corresponding weighted result belongs as the image retrieval result corresponding to the image to be retrieved.

14. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 11 when executing a program stored in a memory.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

16. A computer program product comprising instructions, characterized in that When the computer program product is run on a computer, the computer is enabled to execute the method according to any one of claims 1 to 11.

Images