Image retrieval method, device and electronic equipment

Abstract

The disclosed embodiments provide an image retrieval method, device, and electronic device, belonging to the field of data processing technology. The method comprises: extracting features from an image to be retrieved using a first layer group of a first network model to obtain a first feature image; calculating features of a target image using a second network model to obtain a feature matrix related to the target image; and performing a retrieval operation based on the feature matrix and the first feature image using the second layer group of the first network model, wherein the retrieval operation is used to determine whether the target image is contained in the image to be retrieved. The disclosed solution allows retrieval of a target image without the need for labeled training data, thereby improving the efficiency of image recognition.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to an image retrieval method, device, and electronic device. Background Art

[0002] With the continuous development of Internet technology, video and image products on the Internet are becoming increasingly abundant. Users are no longer limited to watching these contents on TV. They can also search for content of interest on the Internet and watch it. The massive amount of content poses more challenges to the management of video images or pictures.

[0003] Watermarking technology embeds identifying information directly into digital media without affecting the original media's usability or making it difficult to detect or modify. However, it can be identified and recognized by the manufacturer. This hidden information can be used to authenticate content creators and purchasers, transmit confidential information, or determine if the media has been tampered with. Watermarking is an effective method for protecting information security, anti-counterfeiting and traceability, and copyright protection, and is a key branch and research area in the field of information hiding.

[0004] When a large number of pictures contain different types of watermarks, how to find pictures containing a certain type of watermark has become a major problem that troubles people. Summary of the Invention

[0005] In view of this, embodiments of the present disclosure provide an image retrieval method, apparatus, and electronic device, which at least partially solve the problems existing in the prior art.

[0006] In a first aspect, an embodiment of the present disclosure provides an image retrieval method, comprising:

[0007] Using the first layer group of the first network model to perform feature extraction on the image to be retrieved, to obtain a first feature image;

[0008] Performing feature calculation on the target image through the second network model to obtain a feature matrix related to the target image;

[0009] Based on the feature matrix and the first feature image, a retrieval operation is performed using the second layer group of the first network model, where the retrieval operation is used to determine whether the target image is included in the image to be retrieved.

[0010] According to a specific implementation of the embodiment of the present disclosure, extracting features from the image to be retrieved using the first layer group of the first network model includes:

[0011] In the first layer group, a plurality of feature maps are set;

[0012] Based on the multiple feature maps, objects on the image to be retrieved are detected.

[0013] According to a specific implementation of the embodiment of the present disclosure, detecting the object on the image to be retrieved based on the multiple feature maps includes:

[0014] generating a plurality of fixed-size bounding boxes in the first layer group, each bounding box containing a predicted value of an object;

[0015] Performing non-maximum suppression on the bounding box to obtain a final prediction value for the object;

[0016] Based on the final predicted value for the object, the type of the object is determined.

[0017] According to a specific implementation of the embodiment of the present disclosure, extracting features from the image to be retrieved using the first layer group of the first network model includes:

[0018] Additional convolutional layers are added to the first layer group, and the size of the additional convolutional layers decreases layer by layer.

[0019] According to a specific implementation of the embodiment of the present disclosure, extracting features from the image to be retrieved using the first layer group of the first network model includes:

[0020] A plurality of convolution filters are set for the feature layers in the first layer group, where the convolution filters are used to generate prediction values ​​of a fixed size.

[0021] According to a specific implementation of the embodiment of the present disclosure, the step of performing feature calculation on the target image using the second network model to obtain a feature matrix related to the target image includes:

[0022] Obtaining a calculation result of the second network model for the target image;

[0023] The calculation result is used as a feature matrix related to the target image.

[0024] According to a specific implementation of the embodiment of the present disclosure, performing a retrieval operation based on the feature matrix and the first feature image using the second layer group of the first network model includes:

[0025] Using the feature matrix as the convolution kernel of the last convolution layer of the second layer group;

[0026] Performing convolution calculation using the convolution kernel and the feature map output by the second layer group to obtain multiple feature maps;

[0027] Based on the multiple feature maps, a final retrieval result is determined.

[0028] According to a specific implementation of the embodiment of the present disclosure, determining the final search result based on the multiple feature graphs includes:

[0029] determining a plurality of reference windows corresponding to the plurality of feature maps;

[0030] performing classification regression calculation on the multiple reference windows;

[0031] Based on the result of the classification regression calculation, it is determined whether the image to be retrieved contains the target image.

[0032] In a second aspect, an embodiment of the present disclosure provides an image retrieval device, comprising:

[0033] An extraction module, configured to extract features of the image to be retrieved using the first layer group of the first network model to obtain a first feature image;

[0034] a calculation module, configured to perform feature calculation on the target image using a second network model to obtain a feature matrix related to the target image;

[0035] A retrieval module is used to perform a retrieval operation based on the feature matrix and the first feature image using the second layer group of the first network model, wherein the retrieval operation is used to determine whether the target image is included in the image to be retrieved.

[0036] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0037] at least one processor; and,

[0038] a memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the image retrieval method in any of the aforementioned first aspects or any implementation of the first aspect.

[0040] In a fourth aspect, an embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the image retrieval method in the aforementioned first aspect or any implementation of the first aspect.

[0041] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the image retrieval method in the aforementioned first aspect or any implementation of the first aspect.

[0042] The image retrieval scheme in the disclosed embodiment includes extracting features from a target image using the first layer group of a first network model to obtain a first feature image; calculating features of the target image using the second network model to obtain a feature matrix associated with the target image; and performing a retrieval operation based on the feature matrix and the first feature image using the second layer group of the first network model to determine whether the target image is contained in the target image. The disclosed scheme enables retrieval of target images without the need for labeled training data, improving image recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] Figure 1 A schematic diagram of an image retrieval process provided by an embodiment of the present disclosure;

[0045] Figure 2 A schematic diagram of a network model for image retrieval provided in an embodiment of the present disclosure;

[0046] Figure 3 A schematic diagram of another image retrieval process provided by an embodiment of the present disclosure;

[0047] Figure 4 A schematic diagram of another image retrieval process provided by an embodiment of the present disclosure;

[0048] Figure 5 A schematic diagram of the structure of an image retrieval device provided in an embodiment of the present disclosure;

[0049] Figure 6 A schematic diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0051] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0052] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement an apparatus and / or practice a method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this apparatus and / or practice this method.

[0053] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The illustrations only show components related to the present disclosure and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0054] Additionally, in the following description, specific details are provided to provide a thorough understanding of the examples. However, one skilled in the art will appreciate that the aspects described can be practiced without these specific details.

[0055] The present disclosure provides an image retrieval method. The image retrieval method provided in this embodiment can be executed by a computing device, which can be implemented as software or a combination of software and hardware, and can be integrated into a server, terminal device, etc.

[0056] See also Figure 1 , an image retrieval method provided by an embodiment of the present disclosure includes the following steps:

[0057] S101, using the first layer group of the first network model to perform feature extraction on the image to be retrieved to obtain a first feature image.

[0058] Training deep learning models often requires a large amount of training data, but obtaining large amounts of data in practical applications is often difficult. Therefore, how to detect a specific watermark with little or no data is of great practical significance.

[0059] To this end, in the absence of a large number of training samples, the solution disclosed herein utilizes the first network model and the second network model to cooperate with each other to achieve rapid retrieval of the image to be retrieved, thereby improving the efficiency of data retrieval.

[0060] The first network model can be implemented by constructing a neural network model. As an example, the first network model may include a convolution layer, a pooling layer, and a sampling layer.

[0061] The main parameters of a convolutional layer include the size of the convolution kernel and the number of input feature maps. Each convolutional layer can contain several feature maps of the same size. The feature values ​​in the same layer use shared weights, and the convolution kernel size within each layer is consistent. The convolutional layer performs convolution calculations on the input image and extracts the layout features of the input image.

[0062] The feature extraction layer of the convolutional layer can be connected to the sampling layer. The sampling layer is used to calculate the local average of the input image vector and perform secondary feature extraction. By connecting the sampling layer with the convolutional layer, it can ensure that the neural network model has good robustness to the input audio vector.

[0063] In order to speed up the training of the neural network model, a pooling layer is set after the convolution layer. The pooling layer uses the maximum pooling method to process the output results of the convolution layer, which can better extract the invariant features of the input image.

[0064] According to actual needs, multiple convolution layers, pooling layers, and sampling layers can be selected in the first network model to form a first layer group, and the image features on the image to be retrieved are extracted through the first layer group.

[0065] S102: Perform feature calculation on the target image through the second network model to obtain a feature matrix related to the target image.

[0066] The target image is the subject of image retrieval. As an example, the target image can be a watermark pattern, and the image to be retrieved is an image with a watermark. The process of image retrieval is to retrieve a specific watermark image from images with different types of watermarks.

[0067] The target image, as a component of the image to be retrieved, usually occupies a small area in the target image. With a small training sample size, it is often difficult to accurately retrieve the target image from the image to be retrieved. Therefore, a second network model is constructed to directly extract the features of the target image. These features are then used as input to the first network model, allowing for rapid retrieval of the target image.

[0068] The second network model can be implemented by constructing a neural network model. As an example, the second network model can include a convolution layer, a pooling layer, and a sampling layer.

[0069] The main parameters of a convolutional layer include the size of the convolution kernel and the number of input feature maps. Each convolutional layer can contain several feature maps of the same size. The feature values ​​in the same layer use shared weights, and the convolution kernel size within each layer is consistent. The convolutional layer performs convolution calculations on the input image and extracts the layout features of the input image.

[0070] The feature extraction layer of the convolutional layer can be connected to the sampling layer. The sampling layer is used to calculate the local average of the input image vector and perform secondary feature extraction. By connecting the sampling layer with the convolutional layer, it can ensure that the neural network model has good robustness to the input audio vector.

[0071] In order to speed up the training of the neural network model, a pooling layer is set after the convolution layer. The pooling layer uses the maximum pooling method to process the output results of the convolution layer, which can better extract the invariant features of the input image.

[0072] Through the second network model, the target image can be calculated to obtain a feature matrix related to the target object. Through the feature matrix, the target image can be further retrieved.

[0073] S103: Based on the feature matrix and the first feature image, a retrieval operation is performed using the second layer group of the first network model, where the retrieval operation is used to determine whether the image to be retrieved contains the target image.

[0074] After obtaining the feature matrix, the feature matrix is ​​used as input to the first network model. The first feature image is classified and regressed using the feature matrix and the second layer group of the first network model, thereby obtaining a retrieval result for the target image. The second layer group includes at least one convolutional layer from the first network model.

[0075] Specifically, the feature matrix of the target image output by the second network model (containing the information of the target image) is used as the convolution kernel of the last layer of convolution in the first network model, and convolution calculation is performed with the feature map of the second-to-last layer of the first network model to generate the feature map of the last layer. The feature map of the last layer is used to generate a reference window (anchor), and the reference window (anchor) is classified and regressed to realize the detection of the target image (for example, a watermark), thereby realizing the detection of the target image in the absence of target image annotation data.

[0076] Through the above scheme, see Figure 2 The output of the second network model is used as the input to the second layer group of the first network model, and the output of the first layer group is used to jointly retrieve the target image. After the first and second network models are trained, new target images can be detected by simply using the new target images as the input of the second network model without the need for labeled training data.

[0077] See also Figure 3 According to a specific implementation of the embodiment of the present disclosure, the feature extraction of the image to be retrieved using the first layer group of the first network model includes:

[0078] S301: In the first layer group, set a plurality of feature maps.

[0079] In order to improve the retrieval efficiency of the first layer group, multiple feature maps can be set in the first layer group, and these feature maps are used as convolution kernels, thereby improving the feature extraction efficiency of the image to be retrieved.

[0080] S302: Detect objects on the image to be retrieved based on the multiple feature maps.

[0081] Specifically, the process of implementing step S302 may include the following steps:

[0082] S3021, generating a plurality of fixed-size bounding boxes in the first layer group, each bounding box containing a predicted value of an object;

[0083] S3022, performing non-maximum suppression on the bounding box to obtain a final prediction value for the object;

[0084] S3033: Determine the type of the object based on the final predicted value for the object.

[0085] During feature extraction, different convolutional layers can be set in the first layer group of the first network model according to actual needs. For example, according to a specific implementation of the embodiments of the present disclosure, extracting features from the image to be retrieved using the first layer group of the first network model includes adding additional convolutional layers to the first layer group, where the size of the additional convolutional layers decreases layer by layer. By setting additional convolutional layers, more image features can be extracted according to actual needs.

[0086] In addition, according to a specific implementation method of an embodiment of the present disclosure, in the process of extracting features of the image to be retrieved using the first layer group of the first network model, multiple convolution filters can also be set for the feature layer in the first layer group to generate a fixed-size prediction value.

[0087] According to a specific implementation method of an embodiment of the present disclosure, the feature calculation of the target image through the second network model to obtain a feature matrix related to the target image includes: obtaining the calculation result of the second network model for the target image; and using the calculation result as the feature matrix related to the target image.

[0088] See also Figure 4 According to a specific implementation of the embodiment of the present disclosure, performing a retrieval operation based on the feature matrix and the first feature image using the second layer group of the first network model includes:

[0089] S401: Use the feature matrix as the convolution kernel of the last convolution layer of the second layer group.

[0090] By using the feature matrix as the convolution kernel of the last convolution layer of the second layer group, the image to be retrieved can be retrieved accurately and quickly based on the features of the target image.

[0091] S402: Perform convolution calculation using the convolution kernel and the feature map output by the second layer group to obtain multiple feature maps.

[0092] S403: Determine a final retrieval result based on the multiple feature graphs.

[0093] Specifically, the process of implementing step S403 may include the following steps:

[0094] S4031: Determine a plurality of reference windows corresponding to the plurality of feature maps.

[0095] Window convolution kernel calculation is performed using the multiple feature maps obtained in step S403. For example, a 3*3 window convolution kernel can be used for convolution feature calculation. When the convolution kernel slides to a certain position in the feature map, it is mapped to an area on the first feature image centered on the current sliding window center. The center of this area on the first feature image corresponds to a scale and aspect ratio, thus forming multiple reference windows.

[0096] S4031: Perform classification regression calculation on the multiple reference windows.

[0097] A k-means clustering algorithm or a similar classification regression algorithm may be used to perform classification regression calculation on the reference window, and ultimately determine one or several typical reference windows.

[0098] S4031: Based on the result of the classification regression calculation, determine whether the image to be retrieved contains the target image.

[0099] By comparing the similarity between the final regressed reference window and the target image feature matrix, it can be determined whether the image to be retrieved contains the target image. For example, when the similarity is greater than a preset value, it is determined that the image to be retrieved contains the target image.

[0100] Corresponding to the above method embodiment, see Figure 5 The present disclosure further provides an image retrieval device 50, comprising:

[0101] The extraction module 501 is used to extract features of the image to be retrieved using the first layer group of the first network model to obtain a first feature image.

[0102] Training deep learning models often requires a large amount of training data, but obtaining large amounts of data in practical applications is often difficult. Therefore, how to detect a specific watermark with little or no data is of great practical significance.

[0103] To this end, in the absence of a large number of training samples, the solution disclosed herein utilizes the first network model and the second network model to cooperate with each other to achieve rapid retrieval of the image to be retrieved, thereby improving the efficiency of data retrieval.

[0104] The first network model can be implemented by constructing a neural network model. As an example, the first network model may include a convolution layer, a pooling layer, and a sampling layer.

[0105] The main parameters of a convolutional layer include the size of the convolution kernel and the number of input feature maps. Each convolutional layer can contain several feature maps of the same size. The feature values ​​in the same layer use shared weights, and the convolution kernel size within each layer is consistent. The convolutional layer performs convolution calculations on the input image and extracts the layout features of the input image.

[0106] The feature extraction layer of the convolutional layer can be connected to the sampling layer. The sampling layer is used to calculate the local average of the input image vector and perform secondary feature extraction. By connecting the sampling layer with the convolutional layer, it can ensure that the neural network model has good robustness to the input audio vector.

[0107] In order to speed up the training of the neural network model, a pooling layer is set after the convolution layer. The pooling layer uses the maximum pooling method to process the output results of the convolution layer, which can better extract the invariant features of the input image.

[0108] According to actual needs, multiple convolution layers, pooling layers, and sampling layers can be selected in the first network model to form a first layer group, and the image features on the image to be retrieved are extracted through the first layer group.

[0109] The calculation module 502 is used to perform feature calculation on the target image through the second network model to obtain a feature matrix related to the target image.

[0110] The target image is the subject of image retrieval. As an example, the target image can be a watermark pattern, and the image to be retrieved is an image with a watermark. The process of image retrieval is to retrieve a specific watermark image from images with different types of watermarks.

[0111] The target image, as a component of the image to be retrieved, usually occupies a small area in the target image. With a small training sample size, it is often difficult to accurately retrieve the target image from the image to be retrieved. Therefore, a second network model is constructed to directly extract the features of the target image. These features are then used as input to the first network model, allowing for rapid retrieval of the target image.

[0112] The second network model can be implemented by constructing a neural network model. As an example, the second network model can include a convolution layer, a pooling layer, and a sampling layer.

[0113] The main parameters of a convolutional layer include the size of the convolution kernel and the number of input feature maps. Each convolutional layer can contain several feature maps of the same size. The feature values ​​in the same layer use shared weights, and the convolution kernel size within each layer is consistent. The convolutional layer performs convolution calculations on the input image and extracts the layout features of the input image.

[0114] The feature extraction layer of the convolutional layer can be connected to the sampling layer. The sampling layer is used to calculate the local average of the input image vector and perform secondary feature extraction. By connecting the sampling layer with the convolutional layer, it can ensure that the neural network model has good robustness to the input audio vector.

[0115] In order to speed up the training of the neural network model, a pooling layer is set after the convolution layer. The pooling layer uses the maximum pooling method to process the output results of the convolution layer, which can better extract the invariant features of the input image.

[0116] Through the second network model, the target image can be calculated to obtain a feature matrix related to the target object. Through the feature matrix, the target image can be further retrieved.

[0117] The retrieval module 503 is used to perform a retrieval operation based on the feature matrix and the first feature image using the second layer group of the first network model, wherein the retrieval operation is used to determine whether the target image is included in the image to be retrieved.

[0118] After obtaining the feature matrix, the feature matrix is ​​used as input to the first network model. The first feature image is classified and regressed using the feature matrix and the second layer group of the first network model, thereby obtaining a retrieval result for the target image. The second layer group includes at least one convolutional layer from the first network model.

[0119] Specifically, the feature matrix of the target image output by the second network model (containing the information of the target image) is used as the convolution kernel of the last layer of convolution in the first network model, and convolution calculation is performed with the feature map of the second-to-last layer of the first network model to generate the feature map of the last layer. The feature map of the last layer is used to generate a reference window (anchor), and the reference window (anchor) is classified and regressed to realize the detection of the target image (for example, a watermark), thereby realizing the detection of the target image in the absence of target image annotation data.

[0120] Through the above scheme, see Figure 2 The output of the second network model is used as the input to the second layer group of the first network model, and the output of the first layer group is used to jointly retrieve the target image. After the first and second network models are trained, new target images can be detected by simply using the new target images as the input of the second network model without the need for labeled training data.

[0121] Figure 5 The device shown can correspondingly execute the contents of the above method embodiment. For the parts not described in detail in this embodiment, refer to the contents recorded in the above method embodiment and will not be repeated here.

[0122] See also Figure 6 The present disclosure further provides an electronic device 60, which includes:

[0123] at least one processor; and,

[0124] a memory communicatively connected to the at least one processor; wherein,

[0125] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to execute the image retrieval method in the aforementioned method embodiment.

[0126] The embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the aforementioned method embodiment.

[0127] An embodiment of the present disclosure also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, enable the computer to execute the image retrieval method in the aforementioned method embodiment.

[0128] Reference below Figure 6 , which shows a schematic structural diagram of an electronic device 60 suitable for implementing the embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0129] like Figure 6 As shown, the electronic device 60 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 60 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0130] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 60 to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows the electronic device 60 with various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0131] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0132] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0133] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0134] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains at least two Internet Protocol addresses; sends a node evaluation request including the at least two Internet Protocol addresses to a node evaluation device, wherein the node evaluation device selects an Internet Protocol address from the at least two Internet Protocol addresses and returns it; receives the Internet Protocol address returned by the node evaluation device; wherein the obtained Internet Protocol address indicates an edge node in a content distribution network.

[0135] Alternatively, the computer-readable medium carries one or more programs, which, when executed by the electronic device, causes the electronic device to: receive a node evaluation request including at least two Internet Protocol addresses; select an Internet Protocol address from the at least two Internet Protocol addresses; and return the selected Internet Protocol address; wherein the received Internet Protocol address indicates an edge node in a content distribution network.

[0136] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0139] It should be understood that various parts of the present disclosure can be implemented in hardware, software, firmware, or a combination thereof.

[0140] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. An image retrieval method, characterized in that: include: Using the first layer group of the first network model to perform feature extraction on the image to be retrieved, to obtain a first feature image; Performing feature calculation on the target image through the second network model to obtain a feature matrix related to the target image; Based on the feature matrix obtained by the second network model and the first feature image obtained by the first network model, a retrieval operation is performed using the second layer group of the first network model, wherein the retrieval operation is used to determine whether the image to be retrieved contains the target image; Among them, based on the feature matrix and the first feature image, a retrieval operation is performed using the second layer group of the first network model, including: using the feature matrix as the convolution kernel of the last convolution layer of the second layer group; using the convolution kernel and the feature map output by the penultimate layer of the second layer group to perform convolution calculation to obtain multiple feature maps; based on the multiple feature maps, determining the final retrieval result.

2. The method according to claim 1, characterized in that The extracting features of the image to be retrieved by using the first layer group of the first network model includes: In the first layer group, a plurality of feature maps are set; Based on the multiple feature maps, objects on the image to be retrieved are detected.

3. The method according to claim 2, characterized in that The detecting of the object on the image to be retrieved based on the multiple feature maps includes: generating a plurality of fixed-size bounding boxes in the first layer group, each bounding box containing a predicted value of an object; Performing non-maximum suppression on the bounding box to obtain a final prediction value for the object; Based on the final predicted value for the object, the type of the object is determined.

4. The method according to claim 1, wherein The extracting features of the image to be retrieved by using the first layer group of the first network model includes: Additional convolutional layers are added to the first layer group, and the size of the additional convolutional layers decreases layer by layer.

5. The method according to claim 1, wherein The extracting features of the image to be retrieved by using the first layer group of the first network model includes: A plurality of convolution filters are set for the feature layers in the first layer group, where the convolution filters are used to generate prediction values ​​of a fixed size.

6. The method according to claim 1, characterized in that The step of performing feature calculation on the target image by using the second network model to obtain a feature matrix related to the target image includes: Obtaining a calculation result of the second network model for the target image; The calculation result is used as a feature matrix related to the target image.

7. The method according to claim 1, characterized in that Determining a final search result based on the multiple feature graphs includes: determining a plurality of reference windows corresponding to the plurality of feature maps; performing classification regression calculation on the multiple reference windows; Based on the result of the classification regression calculation, it is determined whether the image to be retrieved contains the target image.

8. An image retrieval device, characterized in that: include: An extraction module, configured to extract features of the image to be retrieved using the first layer group of the first network model to obtain a first feature image; a calculation module, configured to perform feature calculation on the target image using a second network model to obtain a feature matrix related to the target image; a retrieval module configured to perform a retrieval operation using the second layer group of the first network model based on the feature matrix obtained by the second network model and the first feature image obtained by the first network model, wherein the retrieval operation is used to determine whether the target image is included in the image to be retrieved; Among them, based on the feature matrix and the first feature image, a retrieval operation is performed using the second layer group of the first network model, including: using the feature matrix as the convolution kernel of the last convolution layer of the second layer group; using the convolution kernel and the feature map output by the penultimate layer of the second layer group to perform convolution calculation to obtain multiple feature maps; based on the multiple feature maps, determining the final retrieval result.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the image retrieval method according to any one of claims 1 to 7. 10 . A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the image retrieval method according to claim 1 .

Images