Image retrieval method and electronic equipment

Abstract

The invention provides an image retrieval method and electronic equipment, and the method comprises the steps: carrying out the first calculation of feature descriptors of all non-central regions of a to-be-retrieved image and feature descriptors of a central point region, obtaining a first feature descriptor calculation value, and obtaining a target feature descriptor of the to-be-retrieved image, and calculating the similarity between the target feature descriptor of the to-be-retrieved image and the target feature descriptor of each sample image in the image data set to obtain an image similarity so as to determine a target image matched with the to-be-retrieved image from the sample images. Thus, on the basis of the mode of determining the target image through calculation of the image feature descriptors, image illumination and seasonal robustness are guaranteed, image retrieval accuracy is improved, and meanwhile the target feature descriptors of the image are obtained through calculation of the feature descriptors of the central point area and the non-central area of the image. When the number of images is large, the calculation amount is reduced, and the image retrieval speed and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of image retrieval technology, and in particular to an image retrieval method and electronic equipment. Background Art

[0002] Image retrieval technology is a computer vision technology that mainly uses the image to be retrieved input by the user to quickly find images that match the image to be retrieved from a large number of sample images in the image data set. In the prior art, image retrieval is usually achieved using image description methods based on local grayscale of pixels and image feature description sub-methods.

[0003] Image retrieval is realized based on the image description method of local grayscale of pixels. Specifically, the image to be retrieved is obtained, a pixel point is selected in the local area of ​​the image to be retrieved, and the grayscale value of the selected pixel point and the grayscale values ​​of the surrounding pixels are calculated. Then, the feature vector of the image to be retrieved is obtained according to the grayscale value of the pixel point, and the feature vectors of multiple sample images in the image data set are determined in the same way. According to the similarity between the feature vector of the image to be retrieved and the feature vectors of each sample image in the image data set, the image matching the image to be retrieved is retrieved from the image data set. When performing image retrieval based on this method, when calculating the grayscale value of the pixel point, the pixel brightness of different areas on the image surface is required to be consistent. Therefore, the image feature matching does not reflect the influence of factors such as lighting and season on the image. The image lighting and seasonal robustness of this method are poor, which affects the accuracy of image retrieval.

[0004] Image retrieval is realized based on the image feature descriptor method, specifically: obtain the image to be retrieved, calculate the gradient value of each pixel in the image to be retrieved, obtain the gradient direction histogram (Histogram of Oriented Gradient, referred to as HOG) according to the gradient value, and then construct the HOG feature descriptor of the image to be retrieved based on HOG, and determine the HOG feature descriptors of multiple sample images in the image data set in the same way. According to the similarity between the HOG feature descriptor of the image to be retrieved and the HOG feature descriptors of each sample image in the image data set, find the image matching the image to be retrieved from the image data set. Image retrieval based on this method can improve the image lighting and seasonal robustness, but it is necessary to calculate the gradient value of each pixel in each image to obtain the corresponding HOG feature descriptor. When the number of images is large, the amount of calculation is large, which affects the image retrieval speed and accuracy.

[0005] In summary, the image retrieval method in the prior art has the problem of poor image retrieval speed and accuracy. Summary of the invention

[0006] The purpose of this application is to solve the problems of poor image retrieval speed and accuracy in the image retrieval methods in the prior art.

[0007] Among them, the feature descriptors corresponding to multiple non-central areas of the image to be retrieved are respectively first calculated with the feature descriptors corresponding to the central point area to obtain a first feature descriptor calculation value, and then the target feature descriptor of the image to be retrieved is obtained based on the first feature descriptor calculation value, so as to calculate the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptors of each sample image in the image data set, and after obtaining the corresponding image similarity, the target image matching the image to be retrieved is determined from the sample images included in the image data set according to the image similarity.

[0008] In this way, based on the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set, the target image matching the image to be retrieved is determined, and the target image does not need to be determined by the grayscale value of the image pixel point, which reduces the influence of factors such as lighting and season on the image, can improve the robustness of image lighting and season, and improve the accuracy of image retrieval. In addition, by calculating the first feature descriptor calculation value of the feature descriptor corresponding to each non-center point area of ​​the image to be retrieved and the feature descriptor corresponding to the center point area, the target feature descriptor of the image to be retrieved is obtained. When the number of images is large, the calculation amount of the feature descriptor obtained by calculating the gradient value of each pixel point in each image can be reduced, thereby improving the speed and accuracy of image retrieval.

[0009] To solve the above technical problems, in a first aspect, an embodiment of the present application discloses an image retrieval method, which includes: determining a feature descriptor corresponding to a center point area of ​​an image to be retrieved and feature descriptors corresponding to multiple non-center point areas; performing a first calculation on the feature descriptors corresponding to each non-center point area and the feature descriptors corresponding to the center point area to obtain a first feature descriptor calculation value; obtaining a target feature descriptor of the image to be retrieved based on the first feature descriptor calculation value; calculating the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptors of each sample image in an image data set to obtain a corresponding image similarity; and determining a target image matching the image to be retrieved from the sample images included in the image data set according to the image similarity.

[0010] By adopting the above technical solution, the feature descriptors corresponding to the multiple non-central areas of the determined image to be retrieved are respectively calculated with the feature descriptors corresponding to the central point area to obtain the first feature descriptor calculation value, and then the target feature descriptor of the image to be retrieved is obtained based on the first feature descriptor calculation value, so as to calculate the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set, and after obtaining the corresponding image similarity, the target image matching the image to be retrieved is determined from the sample images included in the image data set according to the image similarity. In this way, based on the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set, the target image matching the image to be retrieved is determined, and there is no need to determine the target image by calculating the grayscale value of the image pixel point, which reduces the influence of factors such as lighting and season on the image, can improve the robustness of lighting and season, and improve the accuracy of image retrieval. Moreover, by calculating the feature descriptors corresponding to each non-center point area of ​​the image to be retrieved respectively with the feature descriptors corresponding to the center point area, the feature descriptor calculation value is obtained, and then the target feature descriptor of the image to be retrieved is obtained. When the number of images is large, the amount of calculation of the feature descriptor can be reduced by calculating the gradient value of each pixel point in each image to improve the image retrieval speed and accuracy.

[0011] According to another specific embodiment of the present application, an image retrieval method disclosed in the embodiment of the present application, a target feature descriptor of a sample image is obtained by: determining a feature descriptor of the sample image, and determining a feature descriptor of a target sample image related to the sample image; performing weighted averaging processing on the feature descriptor of the sample image and the feature descriptor of the target sample image to obtain the target feature descriptor of the sample image.

[0012] By adopting the above technical solution, the feature descriptor of the sample image in the image data set and the feature descriptor of the target sample image related to the sample image are first determined, and then the feature descriptor of the sample image and the feature descriptor of the target sample image are weighted averaged to obtain the target feature descriptor of the sample image in the image data set. In this way, by performing weighted average processing on the feature descriptor of the sample image in the image data set and the feature descriptor of the target sample image to determine the target feature descriptor of the sample image, the amount of calculation of the target feature descriptor of the image can be reduced when there are a large number of images in the image data set.

[0013] According to another specific embodiment of the present application, an image retrieval method disclosed in the embodiment of the present application, the feature descriptor of a sample image is obtained in the following manner: determining a feature descriptor corresponding to a center point area of ​​the sample image and feature descriptors corresponding to multiple non-center point areas; performing a second calculation on the feature descriptors corresponding to each non-center point area of ​​the sample image and the feature descriptors corresponding to the center point area to obtain a second feature descriptor calculation value; and obtaining a feature descriptor of the sample image based on the second feature descriptor calculation value.

[0014] By adopting the above technical solution, the feature descriptors corresponding to the non-center point areas of the sample image in the determined image data set are respectively subjected to a second calculation with the feature descriptors corresponding to the center point area to obtain a second feature descriptor calculation value, and then the feature descriptor of the sample image is obtained based on the second feature descriptor calculation value. In this way, by calculating the feature descriptors corresponding to the non-center point areas of the sample image in the image data set and the feature descriptors corresponding to the center point area, the feature descriptor of the sample image is determined. When there are a large number of images in the image data set, the calculation amount of the feature descriptor can be reduced by calculating the grayscale value of the pixel points in the image to determine the feature descriptor, thereby improving the speed of image retrieval.

[0015] According to another specific embodiment of the present application, an image retrieval method disclosed in the embodiment of the present application, the target sample images are two sample images adjacent to the sample image in the image generation sequence.

[0016] By adopting the above technical scheme, two sample images adjacent to the sample image in the image generation sequence are determined as target sample images, so as to determine the target feature descriptor of the sample image based on the feature descriptors of the sample image and the target sample image, which can reduce the amount of calculation of the sample images in the image data set and improve the speed and accuracy of image retrieval.

[0017] According to another specific embodiment of the present application, in an image retrieval method disclosed in the embodiment of the present application, the first calculation is a subtraction operation.

[0018] By adopting the above technical scheme, the feature descriptors corresponding to the non-center point areas of the image to be retrieved and the sample image are respectively subtracted from the feature descriptors corresponding to the center point area to obtain a first feature descriptor calculation value, and then the target feature descriptors corresponding to the image to be retrieved and the sample image are obtained based on the first feature descriptor calculation value. When the number of images is large, the speed of image retrieval can be improved by reducing the amount of calculation of the target feature descriptors of the images.

[0019] According to another specific embodiment of the present application, an image retrieval method disclosed in an embodiment of the present application obtains a target feature descriptor of an image to be retrieved based on a calculated value of a first feature descriptor, including: adding all the calculated values ​​of the first feature descriptors obtained to obtain the target feature descriptor of the image to be retrieved.

[0020] By adopting the above technical solution, the target feature descriptor of the image to be retrieved is obtained by adding all the calculated values ​​of the first feature descriptors. In this way, it is only necessary to calculate all the calculated values ​​of the first feature descriptors of the image to be retrieved and then add them to obtain the target feature descriptor of the image to be retrieved, thereby determining the target image that matches the image to be retrieved, which can reduce the amount of calculation of the target feature descriptor and improve the speed of image retrieval.

[0021] According to another specific embodiment of the present application, an embodiment of the present application discloses an image retrieval method, wherein the center point area is the area where the center point of the image to be retrieved is located, and the non-center point area is the grid area of ​​the image to be retrieved after rasterization.

[0022] By adopting the above technical scheme, the area where the center point of the image to be retrieved is located is taken as the center point area, and the grid area of ​​the image to be retrieved after rasterization is taken as the non-center point area, so as to determine the feature descriptor corresponding to the center point area and the feature descriptor corresponding to the non-center point area, the target feature descriptor of the image to be retrieved can be obtained, and then the target image can be determined. There is no need to determine the target image based on the grayscale value of the image pixel points, which can reduce the influence of factors such as lighting and season, and improve the robustness of lighting and season.

[0023] According to another specific embodiment of the present application, an image retrieval method disclosed in the embodiment of the present application, the feature descriptor is a histogram of oriented gradients feature descriptor.

[0024] By adopting the above technical solution, the target image matching the image to be retrieved is found from the image data set through the image directional gradient histogram feature descriptor, which can reduce the impact of factors such as lighting and season on the image and improve lighting and seasonal robustness.

[0025] According to another specific embodiment of the present application, an image retrieval method disclosed in the embodiment of the present application calculates the similarity between a target feature descriptor of an image to be retrieved and a target feature descriptor of each sample image in an image data set, including: performing inner product calculations on the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set, respectively, to obtain the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set.

[0026] By adopting the above technical solution, the target feature descriptor of the image to be retrieved is respectively inner-producted with the target feature descriptor of each sample image in the image data set to obtain the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set. In this way, based on the similarity between the target feature descriptor corresponding to the image to be retrieved and each sample image in the image data set obtained by performing the inner product calculation, the target image is determined, which can improve the accuracy of image retrieval.

[0027] In a second aspect, an embodiment of the present application further discloses an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores a computer program, and the processor executes the computer program stored in the memory, so that the electronic device implements an image retrieval method provided in any one of the embodiments of the first aspect above.

[0028] In a third aspect, an embodiment of the present application further discloses a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the image retrieval method provided in any one of the embodiments of the first aspect above.

[0029] In a fourth aspect, an embodiment of the present application further discloses a computer program product, including a computer program, which, when executed by a processor, implements the image retrieval method provided in any one of the embodiments of the first aspect above.

[0030] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can also be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flowchart of an image retrieval method provided in an embodiment of the present application;

[0032] Figure 2 This is a schematic diagram of a process for obtaining a target feature descriptor of a sample image provided by an embodiment of the present application;

[0033] Figure 3 This is a schematic diagram of a process for obtaining a feature descriptor of a sample image provided by an embodiment of the present application;

[0034] Figure 4 is a schematic diagram of an image retrieval method provided in an embodiment of the present application;

[0035] Figure 5 is an effect diagram of each sample image in the image data set provided in the embodiment of the present application;

[0036] Figure 6It is a rendering of a target image that matches the image to be retrieved and is determined from an image data set provided in an embodiment of the present application;

[0037] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] As mentioned above, image retrieval technology is a computer vision technology that mainly searches for images that match the image to be retrieved from a large number of image data sets based on the image to be retrieved input by the user. In the prior art, image retrieval is usually achieved using an image description method based on local grayscale of pixels and an image feature descriptor method. Among them, when image retrieval is achieved by an image description method based on local grayscale of pixels, the calculation of the grayscale value of the pixel point requires that the grayscale value be calculated when the pixel brightness in different areas of the image surface is consistent, which makes the image feature matching fail to reflect the influence of factors such as illumination and season on the image, resulting in poor robustness of image illumination and season, affecting the accuracy of image retrieval. When image retrieval is achieved by an image feature descriptor-based method, although the HOG feature descriptor of the image can improve the robustness of image illumination and season, it is necessary to calculate the gradient value of each pixel point in each image to obtain the corresponding HOG feature descriptor, so that when the number of images is large, the amount of calculation is large, affecting the speed and accuracy of image retrieval.

[0039] Therefore, the image retrieval method in the prior art has the problem of poor image retrieval speed and accuracy.

[0040] Based on this, the embodiments of the present application disclose an image retrieval method and an electronic device, in which the feature descriptors corresponding to multiple non-central areas of the image to be retrieved are respectively first calculated with the feature descriptors corresponding to the central point area to obtain a first feature descriptor calculation value, and then the target feature descriptor of the image to be retrieved is obtained based on the first feature descriptor calculation value, so as to calculate the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptors of each sample image in the image data set, and obtain the corresponding image similarity, and then determine the target image matching the image to be retrieved from the sample images included in the image data set according to the image similarity.

[0041] In this way, based on the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set, the target image matching the image to be retrieved is determined, and the target image does not need to be determined by the grayscale value of the image pixel point, which reduces the influence of factors such as lighting and season on the image, can improve the robustness of image lighting and season, and improve the accuracy of image retrieval. In addition, by calculating the first feature descriptor calculation value of the feature descriptor corresponding to each non-center point area of ​​the image to be retrieved and the feature descriptor corresponding to the center point area, the target feature descriptor of the image to be retrieved is obtained. When the number of images is large, the calculation amount of the feature descriptor obtained by calculating the gradient value of each pixel point in each image can be reduced, thereby improving the speed and accuracy of image retrieval.

[0042] Next, the image retrieval method provided by the embodiment of the present application is described in detail with reference to the accompanying drawings.

[0043] See also Figure 1 The image retrieval method provided in the embodiment of the present application specifically includes the following steps.

[0044] S100, determining a feature descriptor corresponding to a central point region of an image to be retrieved and feature descriptors corresponding to a plurality of non-central point regions.

[0045] S200, performing a first calculation on the feature descriptors corresponding to each non-central point region and the feature descriptors corresponding to the central point region to obtain a first feature descriptor calculation value.

[0046] S300, obtaining a target feature descriptor of the image to be retrieved based on the calculated value of the first feature descriptor.

[0047] S400, calculating the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set to obtain the corresponding image similarity.

[0048] S500: determining a target image matching the image to be retrieved from sample images included in the image data set according to image similarity.

[0049] The image retrieval method provided in the embodiment of the present application performs a first calculation on the feature descriptors corresponding to the multiple non-central areas of the determined image to be retrieved and the feature descriptors corresponding to the central point area to obtain a first feature descriptor calculation value, and then obtains the target feature descriptor of the image to be retrieved based on the first feature descriptor calculation value, thereby calculating the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptors of each sample image in the image data set, and after obtaining the corresponding image similarity, determine the target image that matches the image to be retrieved from each sample image included in the image data set according to the image similarity.

[0050] In this way, based on the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set, the target image matching the image to be retrieved is determined, and there is no need to determine the target image by calculating the grayscale value of the image pixel points, which reduces the influence of factors such as lighting and season on the image, can improve the robustness of image lighting and season, and improve the accuracy of image retrieval. In addition, by calculating the feature descriptors corresponding to each non-center point area of ​​the image to be retrieved and the feature descriptors corresponding to the center point area, the first feature descriptor calculation value is obtained, and then the target feature descriptor of the image to be retrieved is obtained. When the number of images is large, the amount of calculation of the feature descriptor obtained by calculating the gradient value of each pixel point in each image can be reduced, thereby improving the image retrieval speed and accuracy.

[0051] In one embodiment of the present application, the feature descriptor is a histogram of oriented gradients feature descriptor.

[0052] Exemplarily, with respect to the above-mentioned step S100, determining the feature descriptor corresponding to the center point area of ​​the image to be retrieved and the feature descriptors corresponding to multiple non-center point areas may be to determine the HOG feature descriptor (also known as the histogram of oriented gradients feature descriptor) corresponding to the center point area of ​​the image to be retrieved and the HOG feature descriptors corresponding to multiple non-center point areas, thereby determining the target HOG feature descriptor (also known as the target feature descriptor) of the image to be retrieved.

[0053] Further, in an embodiment of the present application, the center point area is the area where the center point of the image to be retrieved is located, and the non-center point area is the grid area of ​​the image to be retrieved after rasterization.

[0054] Exemplarily, the area where the center point of the image to be retrieved is located is taken as the center point area, and the image to be retrieved is rasterized, and the rasterized grid area of ​​the image to be retrieved is taken as the non-center point area, so that after determining the feature descriptor corresponding to the center point area and the feature descriptor corresponding to the non-center point area, the target feature descriptor of the image to be retrieved can be obtained.

[0055] Further, in one implementation of the present application, the first calculation is a subtraction operation. Of course, the first calculation can also be other operations.

[0056] Exemplarily, regarding the above-mentioned step S200, in one embodiment of the present application, a first calculation is performed on the feature descriptors corresponding to each non-center point area and the feature descriptors corresponding to the center point area to obtain a first feature descriptor calculation value. It can be that after determining the feature descriptor corresponding to the center point area of ​​the image to be retrieved and the feature descriptors corresponding to multiple non-center point areas, the feature descriptors corresponding to the multiple non-center point areas are subtracted from the feature descriptors corresponding to the center point area (that is, the first calculation) to obtain the first feature descriptor calculation value.

[0057] In this way, by performing a first calculation on the HOG feature descriptor (also known as the feature descriptor) corresponding to the center point area of ​​the image to be retrieved and the HOG feature descriptor corresponding to the grid area (also known as the non-center point area) of the rasterized image to be retrieved to obtain a first feature descriptor calculation value, the target feature descriptor of the image to be retrieved is obtained, which can reduce the amount of calculation of the target feature descriptor of the image to be retrieved and improve the speed of image retrieval.

[0058] For example, feature descriptors 1, 2, 3, ..., n corresponding to the non-center point areas of the image to be retrieved and feature descriptor 0 corresponding to the center point area are obtained, and then the first feature descriptor calculation values ​​1-0, 2-0, 3-0, ..., n-0 are obtained.

[0059] Exemplarily, in another embodiment of the present application, after determining the feature descriptor corresponding to the center point area of ​​the image to be retrieved and the feature descriptors corresponding to multiple non-center point areas, the feature descriptors corresponding to the multiple non-center point areas are subtracted from the feature descriptor corresponding to the center point area, and the obtained values ​​are added with the same parameters to obtain the first feature descriptor calculation value; or after adding a coefficient to the feature descriptor corresponding to the center point area, the feature descriptors corresponding to the multiple non-center point areas are subtracted from the feature descriptor corresponding to the center point area to obtain the first feature descriptor calculation value.

[0060] For example, feature descriptors 1, 2, 3, ..., n corresponding to each non-center point area of ​​the image to be retrieved and feature descriptor 0 corresponding to the center point area are obtained, and parameter 1 is obtained, and then the first feature descriptor calculation value 1-0+parameter 1, 2-0+parameter 1, 3-0+parameter 1, ..., n-0+parameter 1 is obtained; or, feature descriptors 1, 2, 3, ..., n corresponding to each non-center point area of ​​the image to be retrieved and feature descriptor 0*coefficient 1 corresponding to the center point area are obtained, and then the first feature descriptor calculation value 1-0*coefficient 1, 2-0*coefficient 1, 3-0*coefficient 1, ..., n-0*coefficient 1 is obtained.

[0061] Furthermore, in one embodiment of the present application, for the above step S300, based on the calculated value of the first feature descriptor, a target feature descriptor of the image to be retrieved is obtained, including: adding all the calculated values ​​of the first feature descriptors obtained to obtain the target feature descriptor of the image to be retrieved.

[0062] Exemplarily, after performing a subtraction operation (i.e., the first calculation) on the feature descriptor corresponding to the center point region of the image to be retrieved and the feature descriptors corresponding to multiple non-center point regions to obtain multiple first feature descriptor calculation values, the multiple first feature descriptor calculation values ​​are added to obtain the target feature descriptor of the image to be retrieved. In this way, the target feature descriptor of the image to be retrieved can be obtained by only subtracting and then adding the feature descriptors corresponding to the center point region and the non-center point region of the image to be retrieved, without the need to calculate the gradient value of each pixel point in the image to be retrieved to obtain the target feature descriptor of the image to be retrieved, thereby reducing the amount of calculation and improving the speed and accuracy of image retrieval.

[0063] Furthermore, in the above step S400, the target feature descriptor of each sample image in the image data set can be obtained in the following manner.

[0064] For example, see Figure 2 In one embodiment of the present application, the target feature descriptor of the sample image is obtained in the following manner.

[0065] S110, determining a feature descriptor of a sample image, and determining a feature descriptor of a target sample image related to the sample image.

[0066] In one implementation of the present application, the target sample images are two sample images that are adjacent to the sample image in the image generation sequence. Of course, the target sample images may also be other images in the image data set.

[0067] Exemplarily, a feature descriptor of a sample image and feature descriptors of two sample images adjacent to the sample image are determined, and then a target feature descriptor of the sample image is determined.

[0068] S120, performing weighted averaging processing on the feature descriptor of the sample image and the feature descriptor of the target sample image to obtain the target feature descriptor of the sample image.

[0069] Exemplarily, the feature descriptor of the sample image and the feature descriptors of two sample images adjacent to the sample image are added and averaged (ie, weighted average processing) to obtain the target feature descriptor of the sample image.

[0070] For example, after determining the feature descriptor of sample image n, the feature descriptor of target sample image n-1, and the feature descriptor of target sample image n+1, the feature descriptor of sample image n is added to the feature descriptor of target sample image n-1 and the feature descriptor of target sample image n+1, and the sum is averaged to obtain the target feature descriptor of the sample image.

[0071] Furthermore, when the target sample image is the first two sample images adjacent to the sample image in the image generation sequence, or the last two sample images adjacent to the sample image, the feature descriptor of the sample image and the feature descriptors of the first two sample images adjacent to the sample image, or the feature descriptors of the last two sample images adjacent to the sample image, are weighted averaged to obtain the target feature descriptor of the sample image.

[0072] For example, after determining the feature descriptor of sample image n, and determining the feature descriptor of target sample image n-1 and the feature descriptor of target sample image n-2, weighted averaging is performed on the feature descriptor of sample image n, the feature descriptor of target sample image n-1 and the feature descriptor of target sample image n-2 to obtain the target feature descriptor of the sample image; or after determining the feature descriptor of target sample image n+1 and the feature descriptor of target sample image n+2, weighted averaging is performed on the feature descriptor of sample image n, the feature descriptor of target sample image n+1 and the feature descriptor of target sample image n+2 to obtain the target feature descriptor of the sample image.

[0073] Further, see Figure 3 In one implementation of the present application, in the above step S110, the feature descriptor of the sample image is determined, and the feature descriptor of the sample image can be obtained in the following manner.

[0074] S111, determining a feature descriptor corresponding to a central point region of a sample image and feature descriptors corresponding to a plurality of non-central point regions.

[0075] Exemplarily, in the implementation of the present application, the area where the center point of the sample image is located is taken as the center point area, and the multiple grid areas of the rasterized sample image are taken as the non-center point areas, so as to determine the feature descriptor corresponding to the center point area of ​​the sample image and the feature descriptors corresponding to the non-center point areas. Therefore, in the present application, the sample image is first rasterized to obtain multiple grid areas of the sample image, and then the feature descriptors of the center point area and the non-center point area of ​​the sample image are determined.

[0076] S112, performing a second calculation on the feature descriptors corresponding to the non-central point regions of the sample image and the feature descriptors corresponding to the central point region to obtain a second feature descriptor calculation value.

[0077] Exemplarily, in one embodiment of the present application, the second calculation may be the same subtraction operation as the first calculation, and of course the second calculation may also be other operations. Therefore, in the present application, after determining the feature descriptor corresponding to the center point region of the sample image and the feature descriptors corresponding to multiple non-center point regions in the image data set, the feature descriptors corresponding to each non-center point region of the sample image are subtracted from the feature descriptors corresponding to the center point region (i.e., the second calculation) to obtain the second feature descriptor calculation value.

[0078] For example, feature descriptors 1, 2, 3, ..., n corresponding to the non-center point areas of the sample image and feature descriptor 0 corresponding to the center point area are obtained, and then the second feature descriptor calculation values ​​1-0, 2-0, 3-0, ..., n-0 are obtained.

[0079] Illustratively, in another embodiment of the present application, after determining the feature descriptor corresponding to the center point area and the feature descriptors corresponding to multiple non-center point areas of the sample image, the feature descriptors corresponding to the multiple non-center point areas are subtracted from the feature descriptor corresponding to the center point area, and the obtained values ​​are added with the same parameters to obtain a calculated value of the second feature descriptor; or after adding a coefficient to the feature descriptor corresponding to the center point area, the feature descriptors corresponding to the multiple non-center point areas are subtracted from the feature descriptor corresponding to the center point area to obtain a calculated value of the second feature descriptor.

[0080] For example, feature descriptors 1, 2, 3, ..., n corresponding to each non-center point area of ​​the sample image and feature descriptor 0 corresponding to the center point area are obtained, and parameter 1 is obtained, and then the second feature descriptor calculation value 1-0+parameter 1, 2-0+parameter 1, 3-0+parameter 1, ..., n-0+parameter 1 is obtained; or, feature descriptors 1, 2, 3, ..., n corresponding to each non-center point area of ​​the sample image and feature descriptor 0*coefficient 1 corresponding to the center point area are obtained, and then the second feature descriptor calculation value 1-0*coefficient 1, 2-0*coefficient 1, 3-0*coefficient 1, ..., n-0*coefficient 1 is obtained.

[0081] S113, obtaining a feature descriptor of the sample image based on the second feature descriptor calculation value.

[0082] Exemplarily, in the implementation mode of the present application, after obtaining multiple second feature descriptor calculation values, all the obtained second feature descriptor calculation values ​​are added together to obtain the feature descriptor of the sample image. In this way, the feature descriptors corresponding to the non-center point areas of the sample image are first subtracted from the feature descriptors corresponding to the center point area (i.e., the second calculation) to obtain multiple second feature descriptor calculation values, and then all the second feature descriptor calculation values ​​are added together to obtain the feature descriptor of the sample image. This can reduce the amount of calculation of the feature descriptors when the number of images in the image data set is large, thereby increasing the image retrieval speed.

[0083] Furthermore, for the non-center point area of ​​the image to be retrieved being the grid area of ​​the image to be retrieved after rasterization, and for the non-center point area of ​​the sample image being the grid area of ​​the rasterized sample image, the grid points can be selected fixedly and evenly in the image to be retrieved and the sample image, and the area where the grid points are located is used as the grid area of ​​the image to be retrieved and the grid area of ​​the sample image, so as to determine that the grid area of ​​the image to be retrieved is the non-center point area corresponding to the image to be retrieved, and the grid area of ​​the sample image is the non-center point area corresponding to the sample image. In this way, the image to be retrieved and the sample image are first rasterized, and then the non-center point area is determined based on the obtained grid area, so that the target feature descriptors corresponding to the image to be retrieved and the sample image are obtained through the determined feature descriptors of the non-center point area and the feature descriptors of the center point area, without considering the influence of the pixel brightness of different areas on the image surface on the image retrieval accuracy, reducing the influence of factors such as illumination and season, and being able to improve the robustness of illumination and season, and improve the accuracy of image retrieval.

[0084] Furthermore, in one embodiment of the present application, for the above-mentioned step S400, the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set is calculated, including: performing inner product calculations on the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set, respectively, to obtain the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set.

[0085] Exemplarily, after determining the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set, the target feature descriptor of the image to be retrieved is respectively calculated with the target feature descriptor of each sample image in the image data set to obtain the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set, thereby finding the target image matching the image to be retrieved from each sample image included in the image data set based on the similarity. In this way, the target image is determined based on the similarity between the target feature descriptor corresponding to the image to be retrieved and each sample image in the image data set obtained by performing the inner product calculation, which can improve the accuracy of image retrieval.

[0086] For example, after determining the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set, the target feature descriptor of the image to be retrieved and the target feature descriptor of the sample image are taken as vectors, and the sum of the products of the corresponding elements of the target feature descriptor vector of the image to be retrieved and the target feature descriptor vector of the sample image is calculated to obtain the corresponding inner product, thereby determining the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptor of the sample image based on the inner product. Of course, the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptor of the sample image can also be calculated by other methods.

[0087] Further, for the above step S500, a target image matching the image to be retrieved is determined from sample images included in the image data set based on image similarity. The similarity between the target feature descriptor of the image to be retrieved and the target feature descriptors of each sample image in the image data set can be obtained by inner product calculation to determine the image similarity, and then the target image matching the image to be retrieved is determined from the sample images included in the image data set based on the image similarity.

[0088] In summary, in one embodiment of the present application, see Figure 4In the image data set (i.e., database) generation stage, firstly, the scattered grid points in the image (i.e., sample image) are selected as HOG centers (i.e., grid points) in a fixed and uniform manner, and the HOG descriptors of the areas where the grid points are located (i.e., the feature descriptors corresponding to the non-center point areas of the sample image) are respectively subtracted from the HOG descriptors of the center point areas of the image (i.e., the feature descriptors corresponding to the center point areas of the sample image) (i.e., the second calculation), and then added to obtain the prior descriptor (i.e., feature descriptor) of the sample image. Then, when it is known that the images in the image data set (i.e., autonomous driving data set) are continuous images, the prior descriptor of the current frame image (i.e., sample image) is averaged with the prior descriptors of the previous and next frame images (i.e., target sample image) (i.e., weighted average processing) to obtain the image descriptor (i.e., target feature descriptor) of the sample image, thereby obtaining the image descriptor database.

[0089] During the database use phase, the scattered grid points in the image (that is, the image to be retrieved) are selected as HOG centers (that is, grid points) in a fixed and uniform manner, and the HOG descriptors of the areas where the grid points are located (that is, the feature descriptors corresponding to the non-center point areas of the image to be retrieved) are subtracted from the HOG descriptors of the center point areas of the image (that is, the feature descriptors corresponding to the center point areas of the image to be retrieved) (that is, the first calculation), and then added to obtain the image descriptor of the image to be retrieved (that is, the target feature descriptor). Then, the inner product calculation is performed between the image descriptor of the image to be retrieved and the image descriptor of each sample image in the image data set to obtain the similarity, and then the sample image with the highest similarity is used as the target image for the final output. Figure 5 As shown, the multiple images in the captured scene are used as images in the image database (that is, each sample image in the image data set). For example, when there are multiple vehicles parked in a parking lot, the multiple vehicles are used as each sample image in the image data set. Figure 6 As shown, after the scene changes, based on the image to be retrieved and the sample images in the image data set, a similar scene image (ie, target image) matching the image to be retrieved is determined from the sample images included in the image data set.

[0090] In an implementation manner of the present application, a rasterization operation is first performed on a sample image in an image data set to obtain a feature descriptor corresponding to a center point region and a feature descriptor corresponding to a non-center point region of the sample image, and then a subtraction and addition operation is performed to obtain a feature descriptor of the sample image, and then a weighted average operation is performed to obtain a target feature descriptor of the sample image, which is stored in an image descriptor database. When performing image retrieval, a rasterization operation is performed on the image to be retrieved to obtain a feature descriptor corresponding to a center point region and a feature descriptor corresponding to a non-center point region of the image to be retrieved, and a subtraction and addition operation is performed to obtain a target feature descriptor of the image to be retrieved, thereby performing an inner product calculation on the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set, and after obtaining the similarity, a target image matching the image to be retrieved is found from each sample image included in the image data set according to the similarity.

[0091] Moreover, in the implementation manner of the present application, after obtaining the prior descriptor (i.e., feature descriptor) of the sample image and the image descriptor (i.e., target feature descriptor) of the image to be retrieved, image normalization processing can be performed based on the idea of ​​Vector of Locally Aggregated Descriptors (VLAD) aggregation technology to achieve the purpose of reducing the impact of factors such as lighting and season on the image.

[0092] The image retrieval method provided in the embodiments of the present application is applied to electronic devices.

[0093] See also Figure 7 , Figure 7 FIG. 1 is a schematic diagram of a structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device may include: a transceiver 121 , a processor 122 , and a memory 123 .

[0094] The processor 122 executes the computer execution instructions stored in the memory, so that the processor 122 executes part of the technical solutions of the image retrieval method in the above embodiment. The processor 122 can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital data processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0095] The memory 123 is connected to the processor 122 via a system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0096] By way of example and not limitation, the memory 123 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 123 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 123 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 123 is a non-volatile solid-state memory. In a particular embodiment, the memory 123 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0097] The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0098] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0099] An embodiment of the present application also provides a chip for executing instructions, which is used to execute the technical solution of the image retrieval method in the above embodiment.

[0100] An embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a processor of an electronic device, the processor of the electronic device executes the technical solution of the image retrieval method of the above embodiment.

[0101] In some possible implementations, various aspects of the method provided in the present application may also be implemented in the form of a program product, which includes a program code. When the program product runs on a processor of an electronic device, the program code is used to enable the processor of the electronic device to execute the steps of the method according to various exemplary implementations of the present application described above in this specification. For example, the electronic device may execute the image retrieval method described in the embodiments of the present application.

[0102] The program product may employ any combination of one or more readable media. The readable medium may be a readable data medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0103] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the image retrieval method in the above embodiment can be implemented.

[0104] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable information processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable information processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable information processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable information processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0107] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0108] It should be noted that, in addition to the implementation methods of the present application described in the above-mentioned specific embodiments, those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Although the description of the present application is introduced in conjunction with the preferred embodiment, this does not mean that the features of this invention are limited to the embodiment. On the contrary, the purpose of introducing the invention in conjunction with the implementation method is to cover other options or modifications that may be extended based on the claims of the present application. In order to provide an in-depth understanding of the present application, the above description contains many specific details, and the present application may also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present application, some specific details will be omitted in the description. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0109] It should be noted that in this specification, similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0110] The terms “first”, “second”, etc. are only used for distinguishing descriptions and should not be understood as indicating or implying relative importance.

[0111] Although the present application has been illustrated and described by referring to certain preferred embodiments of the present application, it should be understood by those skilled in the art that the above contents are further detailed descriptions of the present application in combination with specific embodiments, and it cannot be determined that the specific implementation of the present application is limited to these descriptions. Those skilled in the art may make various changes in form and details, including making several simple deductions or substitutions, without departing from the spirit and scope of the present application.

Claims

1. An image retrieval method, characterized in that: The method comprises: Determine a feature descriptor corresponding to a center point region of the image to be retrieved and feature descriptors corresponding to multiple non-center point regions; Performing a first calculation on the feature descriptors corresponding to the non-central point regions and the feature descriptors corresponding to the central point region to obtain a first feature descriptor calculation value; Based on the calculated value of the first feature descriptor, obtaining a target feature descriptor of the image to be retrieved; Calculating the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set to obtain the corresponding image similarity; A target image matching the image to be retrieved is determined from the sample images included in the image data set according to the image similarity.

2. The image retrieval method according to claim 1, characterized in that: The target feature descriptor of the sample image is obtained by: Determining a feature descriptor of the sample image, and determining a feature descriptor of a target sample image related to the sample image; A weighted average process is performed on the feature descriptor of the sample image and the feature descriptor of the target sample image to obtain the target feature descriptor of the sample image.

3. The image retrieval method according to claim 2, characterized in that: The feature descriptor of the sample image is obtained in the following way: Determine a feature descriptor corresponding to a central point region of the sample image and feature descriptors corresponding to a plurality of non-central point regions; Performing a second calculation on the feature descriptors corresponding to the non-central point regions of the sample image and the feature descriptors corresponding to the central point region to obtain a second feature descriptor calculation value; Based on the calculated value of the second feature descriptor, a feature descriptor of the sample image is obtained.

4. The image retrieval method according to claim 2, characterized in that: The target sample images are two sample images adjacent to the sample image in the image generation sequence.

5. The image retrieval method according to any one of claims 1 to 4, characterized in that: The first calculation is a subtraction operation.

6. The image retrieval method according to any one of claims 1 to 5, characterized in that: Obtaining a target feature descriptor of the image to be retrieved based on the calculated value of the first feature descriptor, including: All the calculated values ​​of the first feature descriptors are added together to obtain the target feature descriptor of the image to be retrieved.

7. The image retrieval method according to any one of claims 1 to 6, characterized in that: The center point area is the area where the center point of the image to be retrieved is located, and the non-center point area is the grid area of ​​the image to be retrieved after rasterization.

8. The image retrieval method according to any one of claims 1 to 7, characterized in that: The feature descriptor is a histogram of oriented gradients feature descriptor.

9. The image retrieval method according to any one of claims 1 to 8, characterized in that: Calculating the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptor of each sample image in the image data set, including: The target feature descriptor of the image to be retrieved is respectively inner-product-calculated with the target feature descriptors of each of the sample images in the image data set to obtain the similarity between the target feature descriptor of the image to be retrieved and the target feature descriptors of each of the sample images in the image data set.

10. An electronic device, characterized in that: include: A memory for storing a computer program, wherein the computer program includes program instructions; The processor is used to execute program instructions so that the electronic device executes the image retrieval method as described in any one of claims 1 to 9.

Images