Image retrieval method and image retrieval device

Abstract

The present disclosure provides an image retrieval method and an image retrieval device. The image retrieval device processes the image to be retrieved using a convolution processing layer to obtain a first processing result; processes the first processing result using a fully connected processing layer to obtain a second processing result; processes the second processing result using a feature integration layer to obtain image features; and performs retrieval using the image features to obtain a corresponding target image; wherein the fully connected processing layer includes a plurality of fully connected layers, and the second processing result includes the output results of a predetermined number of neurons in each fully connected layer. The present disclosure effectively reduces the amount of data processing in the feature integration layer and effectively improves the efficiency of image retrieval.

Description

Technical Field

[0001] The present disclosure relates to the field of information processing, and in particular to an image retrieval method and an image retrieval device. Background Art

[0002] With the continuous development of society and the decreasing cost of digital information collection tools and storage devices, daily life is filled with massive multimedia image data, such as e-commerce product images, architectural design images, news and entertainment digital media images, wedding photography images, teaching guidance images, etc. How to efficiently and accurately obtain the image information we need from a large amount of data, that is, image retrieval technology, has a wide range of practical value and significance in life.

[0003] At present, the commonly used technology in image retrieval technology is content-based image retrieval, which extracts valuable image feature vectors from images for subsequent distance calculation and matching recognition, thereby achieving the purpose of retrieval. For example, methods such as SIFT (Scale Invariant Feature Transform), SURF (Speed ​​Up Robust Feature), and HOG (Histogram of Oriented Gradient) all obtain gradient features or other image feature vectors from images, and then use the BoW (Bag-of-Words) model to express the image as a whole, and then retrieve the corresponding target image through distance calculation and image matching. Summary of the invention

[0004] Through research, the inventors found that the presence of a large number of neurons in traditional neural networks results in low computational efficiency of the neural networks, making them incapable of being widely promoted in actual scenarios.

[0005] To this end, the present disclosure provides a solution for effectively improving image retrieval efficiency.

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided an image retrieval method, comprising: processing an image to be retrieved using a convolution processing layer to obtain a first processing result; processing the first processing result using a fully connected processing layer to obtain a second processing result; processing the second processing result using a feature integration layer to obtain image features; and retrieving using the image features to obtain a corresponding target image; wherein the fully connected processing layer comprises a plurality of fully connected layers, and the second processing result comprises the output results of a predetermined number of neurons in each fully connected layer.

[0007] In some embodiments, processing the first processing result using the fully-connected processing layer includes: performing a fully-connected process on the first processing result using the first fully-connected layer; performing a fully-connected process on the output of the (i - 1)-th fully-connected layer using the i-th fully-connected layer, where 1 < i < N and N is the number of fully-connected layers in the fully-connected processing layer; performing a fully-connected process on the output of the (N - 1)-th fully-connected layer using the N-th fully-connected layer; in each fully-connected layer, sending the outputs of a predetermined number of randomly selected neurons to each neuron in the feature integration layer.

[0008] In some embodiments, in each fully-connected layer, sending the outputs of a predetermined number of randomly selected neurons to each neuron in the feature integration layer includes: in each fully-connected layer, determining the number of neurons w to be discarded; generating a random integer r in the range from 1 to n to determine the corresponding neuron a r , where n is the number of neurons in the fully-connected layer; if the neuron a r does not appear in the discard list, then adding the neuron a r to the discard list; if the number of neurons in the discard list has not reached the number of neurons w, then repeating the step of generating a random integer r in the range from 1 to N; if the number of neurons in the discard list has reached the number of neurons w, then sending the outputs of the neurons in the fully-connected layer that are not included in the discard list to each neuron in the feature integration layer.

[0009] In some embodiments, in each fully-connected layer, determining the number of neurons w to be discarded includes: in each fully-connected layer, determining the number of neurons m participating in the selection according to the number of neurons n in the fully-connected layer and a preset selection probability p; determining the number of neurons w to be discarded according to the number of neurons n in the fully-connected layer and the number of neurons m.

[0010] In some embodiments, using to determine the number of neurons m participating in the selection, where is the floor function; using n - m to determine the number of neurons w to be discarded.

[0011] In some embodiments, processing the second processing result using the feature integration layer includes: using each neuron in the feature integration layer to perform an integration process according to the weighted average of each data item in the second processing result.

[0012] In some embodiments, training data is input into a convolutional processing layer to obtain corresponding image features; the obtained image features are used for retrieval to obtain corresponding target images; according to the deviation between the obtained target images and the training images corresponding to the training data, the parameters of the convolutional processing layer, the fully connected processing layer, and the feature integration layer are adjusted so that the deviation is within a predetermined range.

[0013] According to a second aspect of the embodiments of the present disclosure, there is provided an image retrieval device, including: a convolutional processing module configured to process an image to be retrieved using a convolutional processing layer to obtain a first processing result; a fully connected processing module configured to process the first processing result using a fully connected processing layer to obtain a second processing result, where the fully connected processing layer includes a plurality of fully connected layers, and the second processing result includes the output results of a predetermined number of neurons in each fully connected layer; a feature integration processing module configured to process the second processing result using a feature integration layer to obtain image features; and a retrieval module configured to perform retrieval using the image features to obtain corresponding target images.

[0014] In some embodiments, the fully connected processing module is configured to perform a fully connected process on the first processing result using the first fully connected layer, perform a fully connected process on the output of the (i - 1)-th fully connected layer using the i-th fully connected layer, where 1 < i < N and N is the number of fully connected layers in the fully connected processing layer, perform a fully connected process on the output of the (N - 1)-th fully connected layer using the N-th fully connected layer, and in each fully connected layer, send the outputs of a predetermined number of randomly selected neurons to each neuron in the feature integration layer.

[0015] In some embodiments, the fully connected processing module is configured to determine the number of discarded neurons w in each fully connected layer, generate a random integer r in the range of 1 to n to determine the corresponding neuron a r , where n is the number of neurons in the fully connected layer. If the neuron a r does not appear in the discard list, then the neuron a r is included in the discard list. If the number of neurons in the discard list does not reach the number of neurons w, the operation of generating a random integer r in the range of 1 to N is repeated. If the number of neurons in the discard list has reached the number of neurons w, the outputs of the neurons in the fully connected layer that are not included in the discard list are sent to each neuron in the feature integration layer.

[0016] In some embodiments, the fully connected processing module is configured to determine, in each fully connected layer, the number of neurons m participating in the selection according to the number of neurons n in the fully connected layer and a preset selection probability p, and to determine the number of neurons w to be discarded according to the number of neurons n in the fully connected layer and the number of neurons m.

[0017] In some embodiments, the fully connected processing module is configured to utilize Determine the number of neurons m involved in the selection, To round down the function, nm is used to determine the number of neurons w that are discarded.

[0018] In some embodiments, the feature integration processing module is configured to use each neuron in the feature integration layer to perform integration processing according to the weighted average of each data item in the second processing result.

[0019] In some embodiments, the above-mentioned device also includes: a training module, which is configured to input training data into the convolution processing layer to obtain corresponding image features, use the obtained image features to perform retrieval to obtain the corresponding target image, and adjust the parameters of the convolution processing layer, the fully connected processing layer and the feature integration layer according to the deviation between the obtained target image and the training image corresponding to the training data so that the deviation is within a predetermined range.

[0020] According to a third aspect of an embodiment of the present disclosure, there is provided an image retrieval device, comprising: a memory configured to store instructions; a processor coupled to the memory, the processor being configured to execute a method as described in any of the above embodiments based on the instructions stored in the memory.

[0021] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method involved in any of the above embodiments is implemented.

[0022] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1 A schematic diagram of a flow chart of an image retrieval method according to an embodiment of the present disclosure;

[0025] Figure 2 A schematic diagram of the structure of a fully connected processing layer according to an embodiment of the present disclosure;

[0026] Figure 3 A schematic diagram of a neuron selection process according to an embodiment of the present disclosure;

[0027] Figure 4 A schematic diagram of the structure of an image retrieval device according to an embodiment of the present disclosure;

[0028] Figure 5 A schematic diagram of the structure of an image retrieval device according to another embodiment of the present disclosure;

[0029] Figure 6 The figure is a schematic diagram of the structure of an image retrieval device according to another embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0031] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0032] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0033] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0034] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0035] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0036] Figure 1Schematic flowchart of an image retrieval method according to an embodiment of the present disclosure. In some embodiments, the steps of the image retrieval method may be executed by an image retrieval device.

[0037] In step 101, the image to be retrieved is processed using a convolutional processing layer to obtain a first processing result.

[0038] In some embodiments, the convolutional processing layer includes a plurality of convolutional layers, and a pooling layer is configured after each convolutional layer to perform pooling processing on the convolutional result, thereby reducing the data volume of the convolutional result.

[0039] In step 102, the first processing result is processed using a fully connected processing layer to obtain a second processing result.

[0040] It should be noted here that the fully connected processing layer includes a plurality of fully connected layers, and the second processing result includes the output results of a predetermined number of neurons in each fully connected layer.

[0041] In some embodiments, the first fully connected layer is used to perform a fully connected process on the first processing result. The i-th fully connected layer is used to perform a fully connected process on the output of the (i - 1)-th fully connected layer, where 1 < i < N and N is the number of fully connected layers in the fully connected processing layer. The N-th fully connected layer is used to perform a fully connected process on the output of the (N - 1)-th fully connected layer. In each fully connected layer, the outputs of a randomly selected predetermined number of neurons are sent to the neurons in the feature integration layer.

[0042] Figure 2 Schematic structural diagram of a fully connected processing layer according to an embodiment of the present disclosure. As Figure 2 shown, the fully connected processing layer includes 3 fully connected layers.

[0043] In the first fully connected layer, neurons a1 to a4 are respectively fully connected to neurons b1 to b4 in the second fully connected layer. In addition, neurons a2 to a4 are respectively fully connected to neurons d1 to d4 in the feature integration layer.

[0044] In the second fully connected layer, neurons b1 to b4 are respectively fully connected to neurons c1 to c4 in the third fully connected layer. In addition, neurons b1, b3, and b4 are respectively fully connected to neurons d1 to d4 in the feature integration layer.

[0045] In the third fully connected layer, neurons c1, c2, and b4 are respectively fully connected to neurons d1 to d4 in the feature integration layer.

[0046] In step 103, the second processing result is processed using the feature integration layer to obtain image features.

[0047] In some embodiments, each neuron in the feature integration layer is used to perform integration processing according to the weighted average of each data item in the second processing result.

[0048] In step 104, a search is performed using the image features to obtain a corresponding target image.

[0049] In the image retrieval method provided by the above embodiment of the present disclosure, the fully connected processing layer includes a plurality of fully connected layers, and the second processing result includes the output results of a predetermined number of neurons in each fully connected layer. Since only the input results of some neurons in each fully connected layer are sent to the feature integration layer, the data processing amount of the feature integration layer is effectively reduced, and the image retrieval efficiency is effectively improved.

[0050] Figure 3 A flowchart of neuron selection according to an embodiment of the present disclosure is shown in FIG. 1. In some embodiments, the steps of the neuron selection method may be performed by an image retrieval device.

[0051] In step 301, in each fully connected layer, the number of neurons w to be discarded is determined.

[0052] In some embodiments, in each fully connected layer, the number of neurons m participating in the selection is determined according to the number of neurons n in the fully connected layer and the preset selection probability p. The number of neurons w discarded is determined according to the number of neurons n and the number of neurons m in the fully connected layer.

[0053] For example, using Determine the number of neurons m involved in the selection, is the floor rounding function. In addition, nm is used to determine the number of neurons w to be discarded.

[0054] In step 302, a random integer r is generated in the range of 1 to n to determine the corresponding neuron a r , n is the number of neurons in the fully connected layer.

[0055] In step 303, determine whether neuron a r Whether to appear in the discard list.

[0056] It should be noted here that, in the initial state, the discard list is empty.

[0057] If neuron a r does not appear in the discard list, then execute step 304; if neuron a r If it has appeared in the discard list, step 302 is repeated.

[0058] In step 304, neuron a r Included in discard list.

[0059] In step 305, it is determined whether the number of neurons in the discarded list reaches the number of neurons w.

[0060] If the number of neurons in the discarded list has not reached the number of neurons w, step 302 is repeated; if the number of neurons in the discarded list has reached the number of neurons w, step 306 is executed.

[0061] Step 306, sending the output of the neurons in the fully connected layer that are not included in the discard list to each neuron in the feature integration layer.

[0062] For example, Figure 2 As shown, each fully connected layer has 4 neurons, and the preset selection probability p is 0.8. It can be seen that the number of neurons involved in the selection is m. That is, the number of neurons discarded is 4-3=1.

[0063] For the first fully connected layer, a random integer r is obtained in the interval [1,4] using a conventional random algorithm. For example, if r=1, the corresponding neuron is a1. Since the current discard list is empty, a1 is put into the discard list. Since the number of neurons in the current discard list is 1, which is the same as the number of discarded neurons w, the processing is stopped. That is, in the first fully connected layer, neuron a1 is discarded, and only neurons a2 to a4 are fully connected to neurons d1 to d4 in the feature integration layer.

[0064] In the same way, neuron b2 is discarded in the second fully connected layer, and neuron c3 is discarded in the third fully connected layer.

[0065] In some embodiments, Figure 2 As shown, let the outputs of the first fully connected layers a2 to a4 be x1, x2, and x3, respectively. Let the outputs of the second fully connected layers b1, b3, and b4 be x4, x5, and x6, respectively. Let the outputs of the third fully connected layers c1, c2, and c4 be x7, x8, and x9, respectively. The corresponding values ​​of the neurons d1 to d4 in the feature integration layer are y1 to y4, respectively.

[0066] Assume that the corresponding weight parameters of x1 and y1 to y4 are w 11 tow 14 , let the corresponding weight parameters of x2 and y1 to y4 be w 21 tow 24 , let the corresponding weight parameters of x3 and y1 to y4 be w 31 tow 34 Similarly, x4 to x9 and y1 to y4 are also configured with corresponding weight parameters.

[0067] like Figure 2As shown, the connection weights of x1 to x9 and y1 are w 11 tow 91 . Then the corresponding fusion method is:

[0068] g(x1×w 11 +x2×w 21 +x3×w 31 +x4×w 41 +x5×w 51 +x6×w 61 +x7×w 71 +x8×w 81 +x9×w 91 +bias1)=y1

[0069] Where g(.) is the activation function and bias1 is the bias parameter.

[0070] In some embodiments, the activation function is as follows.

[0071]

[0072] Similarly, the connection weights of x1 to x9 and y2 are w 12 tow 92 , then the corresponding fusion method is:

[0073] g(x1×w 12 +x2×w 22 +x3×w 32 +x4×w 42 +x5×w 52 +x6×w 62 +x7×w 72 +x8×w 82 +x9×w 92 + bias2) = y2

[0074] In addition, the connection weights of x1 to x9 and y3 are w 13 tow 93 , then the corresponding fusion method is:

[0075] g(x1×w 13 +x2×w 23 +x3×w 33 +x4×w 43 +x5×w 53 +x6×w 63 +x7×w 73 +x8×w 83 +x9×w 93 + bias3) = y3

[0076] The connection weights of x1 to x9 and y4 are w14 tow 94 , then the corresponding fusion method is:

[0077] g(x1×w 14 +x2×w 24 +x3×w 34 +x4×w 44 +x5×w 54 +x6×w 64 +x7×w 74 +x8×w 84 +x9×w 94 +bias4)=y4

[0078] Where bias2 to bias4 are the corresponding bias parameters.

[0079] The above four fusion formulas are expressed in matrix form as follows:

[0080]

[0081] Therefore, by calculating the output values ​​of the selected 9 neurons through weight parameters and bias values, and mapping them through the activation function g(.), a 4-dimensional vector [y1, y2, y3, y4] is obtained, which is the feature vector output by the feature integration layer.

[0082] It should be noted here that when p is 0.8, the three fully connected layers have only a 0.2×0.2×0.2=0.004 probability of losing information. Considering that the lost information may be existing duplicate information, at least 99.6% of the information in the three fully connected layers can be integrated into the feature integration layer. In this way, valuable information will not be lost, and the amount of calculation can be effectively reduced. Of course, the p value can be optimized and adjusted as needed.

[0083] In some embodiments, the training data is input into the convolution processing layer to obtain the corresponding image features. The obtained image features are used for retrieval to obtain the corresponding target image. According to the deviation between the obtained target image and the training image corresponding to the training data, the parameters of the convolution processing layer, the fully connected processing layer and the feature integration layer are adjusted so that the deviation is within a predetermined range.

[0084] For example, by using the ILSVRC-2012 image library for training. In order to improve the universality of image recognition and retrieval, 25,000 images are randomly obtained from Flickr to form an interference image library. The interference image library contains a variety of images, which contain extremely extensive and rich semantic content for universal training.

[0085] Figure 4The structural schematic diagram of an image retrieval device according to an embodiment of the present disclosure. As Figure 4 shown, the image retrieval device includes a convolution processing module 41, a fully connected processing module 42, a feature integration processing module 43, and a retrieval module 44.

[0086] The convolution processing module 41 is configured to process the image to be retrieved by using a convolution processing layer to obtain a first processing result.

[0087] In some embodiments, the convolution processing layer includes multiple convolution layers, and a pooling layer is configured after each convolution layer to perform pooling processing on the convolution result, thereby reducing the data volume of the convolution result.

[0088] The fully connected processing module 42 is configured to process the first processing result by using a fully connected processing layer to obtain a second processing result, where the fully connected processing layer includes multiple fully connected layers, and the second processing result includes the output results of a predetermined number of neurons in each fully connected layer.

[0089] In some embodiments, the fully connected processing module 42 is configured to perform a fully connected processing on the first processing result by using the first fully connected layer, perform a fully connected processing on the output of the (i - 1)-th fully connected layer by using the i-th fully connected layer, 1 < i < N, where N is the number of fully connected layers in the fully connected processing layer, perform a fully connected processing on the output of the (N - 1)-th fully connected layer by using the N-th fully connected layer, and in each fully connected layer, send the output of a randomly selected predetermined number of neurons to each neuron in the feature integration layer. The corresponding schematic diagram is as Figure 2 shown.

[0090] The feature integration processing module 43 is configured to process the second processing result by using a feature integration layer to obtain image features.

[0091] In some embodiments, the feature integration processing module 43 is configured to perform an integration processing by using each neuron in the feature integration layer according to the weighted average of each data item in the second processing result.

[0092] The retrieval module 44 is configured to perform a retrieval by using the image features to obtain a corresponding target image.

[0093] In the image retrieval device provided in the above embodiments of the present disclosure, the fully connected processing layer includes multiple fully connected layers, and the second processing result includes the output results of a predetermined number of neurons in each fully connected layer. Since only the input results of some neurons in each fully connected layer are sent to the feature integration layer, the data processing volume of the feature integration layer is effectively reduced, and the image retrieval efficiency is effectively improved.

[0094] In some embodiments, the fully connected processing module 42 is configured to determine the number of neurons to be discarded in each fully connected layer, generate a random integer r in the interval from 1 to , to determine the corresponding neuron a r , n is the number of neurons in the fully connected layer, if neuron a r If it does not appear in the discard list, neuron a r If the number of neurons in the discard list does not reach the number of neurons w, the operation of generating a random integer r in the interval of 1 to N is repeated. If the number of neurons in the discard list has reached the number of neurons w, the output of the neurons in the fully connected layer that are not included in the discard list is sent to each neuron in the feature integration layer.

[0095] In some embodiments, the fully connected processing module 42 is configured to determine, in each fully connected layer, the number of neurons m participating in the selection according to the number of neurons n in the fully connected layer and a preset selection probability p, and to determine the number of neurons w to be discarded according to the number of neurons n and the number of neurons m in the fully connected layer.

[0096] For example, the full connection processing module 42 is configured to use Determine the number of neurons m involved in the selection, To round down the function, nm is used to determine the number of neurons w that are discarded.

[0097] Figure 5 It is a structural schematic diagram of an image retrieval device according to another embodiment of the present disclosure.

[0098] like Figure 5 As shown, the image retrieval device further includes a training module 45. The training module 45 is configured to input the training data into the convolution processing layer to obtain the corresponding image features, use the obtained image features to perform retrieval to obtain the corresponding target image, and adjust the parameters of the convolution processing layer, the fully connected processing layer and the feature integration layer according to the deviation between the obtained target image and the training image corresponding to the training data so as to make the deviation within a predetermined range.

[0099] Figure 6 FIG. 1 is a schematic diagram of the structure of an image retrieval device according to another embodiment of the present disclosure. Figure 6 As shown, the image retrieval device includes a memory 61 and a processor 62 .

[0100] The memory 61 is used to store instructions. The processor 62 is coupled to the memory 61. The processor 62 is configured to execute the instructions stored in the memory to implement the following. Figure 1 or Figure 3 The method of any one of the embodiments.

[0101] like Figure 6As shown, the image retrieval device further includes a communication interface 63 for information exchange with other devices. At the same time, the device also includes a bus 64, through which the processor 62, the communication interface 63, and the memory 61 communicate with each other.

[0102] The memory 61 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory. The memory 61 may also be a memory array. The memory 61 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules.

[0103] In addition, the processor 62 may be a central processing unit (CPU), or may be an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present disclosure.

[0104] The present disclosure also relates to a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, which are executed by a processor to implement the following Figure 1 or Figure 3 The method of any one of the embodiments.

[0105] In the present disclosure, since only the input results of some neurons in each fully connected layer are sent to the feature integration layer, the data processing amount of the feature integration layer is effectively reduced, the image retrieval efficiency is effectively improved, and it is convenient for wide promotion and use in actual scenarios.

[0106] In some embodiments, the functional unit module described above can be implemented as a general-purpose processor, a programmable logic controller (PLC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components or any appropriate combination thereof for performing the functions described in the present disclosure.

[0107] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0108] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. An image retrieval method, comprising: Processing the image to be retrieved using a convolutional processing layer to obtain a first processing result; Processing the first processing result using a fully-connected processing layer to obtain a second processing result; Processing the second processing result using a feature integration layer to obtain image features; Performing retrieval using the image features to obtain corresponding target images; wherein the fully-connected processing layer includes a plurality of fully-connected layers, and the second processing result includes output results of a predetermined number of neurons in each fully-connected layer; wherein processing the first processing result using a fully-connected processing layer includes: Performing a fully-connected process on the first processing result using the first fully-connected layer; Performing a fully-connected process on the output of the (i - 1)-th fully-connected layer using the i-th fully-connected layer, where 1 < i < N and N is the number of fully-connected layers in the fully-connected processing layer; Performing a fully-connected process on the output of the (N - 1)-th fully-connected layer using the N-th fully-connected layer; In each fully-connected layer, sending the outputs of a predetermined number of randomly selected neurons to each neuron in the feature integration layer.

2. The method according to claim 1, wherein, in each fully-connected layer, sending the outputs of a predetermined number of randomly selected neurons to each neuron in the feature integration layer includes: Determining the number w of neurons to be discarded in each fully-connected layer; Generate a random integer r in the range of 1 to n to determine the corresponding neuron a r , n is the number of neurons in the fully connected layer; If the neuron a r does not appear in the discard list, then the neuron a r Included in the discard list; If the number of neurons in the discard list has not reached the number w of neurons, repeating the step of generating a random integer r in the range from 1 to n; If the number of neurons in the discard list has reached the number w of neurons, sending the outputs of the neurons in the fully-connected layer that are not included in the discard list to each neuron in the feature integration layer.

3. The method according to claim 2, wherein, in each fully-connected layer, determining the number w of neurons to be discarded includes: In each fully-connected layer, determining the number m of neurons participating in the selection according to the number n of neurons in the fully-connected layer and a preset selection probability p; Determining the number w of neurons to be discarded according to the number n of neurons in the fully-connected layer and the number m of neurons.

4. The method according to claim 3, wherein use Determine the number of neurons m involved in the selection, is the floor rounding function; Determining the number w of neurons to be discarded using n - m.

5. The method according to claim 1, wherein processing the second processing result using a feature integration layer includes: Using each neuron in the feature integration layer to perform an integration process according to the weighted average of each data item in the second processing result.

6. The method according to any one of claims 1 - 5, wherein Inputting training data into the convolutional processing layer to obtain corresponding image features; Performing retrieval using the obtained image features to obtain corresponding target images; Adjusting the parameters of the convolutional processing layer, the fully-connected processing layer, and the feature integration layer according to the deviation between the obtained target image and the training image corresponding to the training data, so that the deviation is within a predetermined range.

7. An image retrieval apparatus, comprising: The convolution processing module is configured to process the image to be retrieved by using a convolution processing layer to obtain a first processing result; The fully connected processing module is configured to process the first processing result by using a fully connected processing layer to obtain a second processing result. Wherein, the fully connected processing layer includes a plurality of fully connected layers, and the second processing result includes the output results of a predetermined number of neurons in each fully connected layer. Specifically, the first fully connected layer is used to perform a fully connected process on the first processing result, the i-th fully connected layer is used to perform a fully connected process on the output of the (i - 1)-th fully connected layer, where 1 < i < N and N is the number of fully connected layers in the fully connected processing layer, and the N-th fully connected layer is used to perform a fully connected process on the output of the (N - 1)-th fully connected layer. In each fully connected layer, the outputs of a randomly selected predetermined number of neurons are sent to each neuron in the feature integration layer; The feature integration processing module is configured to process the second processing result by using a feature integration layer to obtain image features; The retrieval module is configured to perform retrieval by using the image features to obtain corresponding target images.

8. The apparatus according to claim 7, wherein, The fully connected processing module is configured to determine the number of neurons w to be discarded in each fully connected layer, and generate a random integer r in the range of 1 to n to determine the corresponding neuron a r , n is the number of neurons in the fully connected layer, if the neuron a r does not appear in the discard list, then the neuron a r If the number of neurons in the discard list does not reach the number of neurons w, the operation of generating a random integer r in the interval of 1 to N is repeatedly performed; if the number of neurons in the discard list has reached the number of neurons w, the output of the neurons in the fully connected layer that are not included in the discard list is sent to each neuron in the feature integration layer.

9. The apparatus according to claim 8, wherein, The fully connected processing module is configured to, in each fully connected layer, determine the number of neurons m participating in the selection according to the number of neurons n in the fully connected layer and a preset selection probability p, and determine the number of discarded neurons w according to the number of neurons n in the fully connected layer and the number of neurons m.

10. The apparatus according to claim 9, wherein, The fully connected processing module is configured to utilize Determine the number of neurons m involved in the selection, To round down the function, nm is used to determine the number of neurons w that are discarded.

11. The apparatus according to claim 7, wherein, The feature integration processing module is configured to perform integration processing by using each neuron in the feature integration layer according to the weighted average of each data item in the second processing result.

12. The apparatus according to any one of claims 7 - 11 further includes: The training module is configured to input training data into the convolution processing layer to obtain corresponding image features, perform retrieval by using the obtained image features to obtain corresponding target images, and adjust the parameters of the convolution processing layer, the fully connected processing layer, and the feature integration layer according to the deviation between the obtained target images and the training images corresponding to the training data, so that the deviation is within a predetermined range.

13. An image retrieval apparatus includes: A memory configured to store instructions; A processor coupled to the memory, the processor being configured to execute a method as described in any one of claims 1 - 6 based on the instructions stored in the memory.

14. A computer-readable storage medium, wherein: A computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, a method as described in any one of claims 1 - 6 is implemented.

Images