Picture retrieval method and device, equipment and storage medium

Abstract

The invention discloses a picture retrieval method, device and equipment and a storage medium, and is applied to the technical field of big data, and the picture retrieval method comprises the following steps: mapping a query object into a query data point in an embedding space; converting the query data points into query vectors in the embedding space, and converting sample data points in the retrieval database into sample vectors; determining a target sample vector corresponding to the query vector, and obtaining a target sample picture corresponding to the target sample vector; and taking the target sample picture as a picture retrieval result of the query object. Sample pictures are stored in a retrieval database in the form of sample data points, so that the storage pressure of the database is relieved, and when picture retrieval is carried out, whether pictures or picture description characters are input by a user, the pictures or the picture description characters are converted into meaningful query data points, retrieval query is carried out in the form of the data points, and the user experience is improved. Therefore, the inaccurate image retrieval result caused by the influence of factors such as noise in the image is avoided.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to an image retrieval method, device, equipment and storage medium. Background Art

[0002] Image retrieval is a retrieval task. The goal is to retrieve a target image that matches the query object from a database given a query object, such as a picture or a picture description. With the development of Internet technology and the massive generation of digital pictures, image retrieval has been widely used in various fields.

[0003] However, when performing image retrieval, it is necessary to perform queries based on text or images in a retrieval database that stores a large number of images. This not only increases the storage pressure of the database, but also is easily affected by factors such as noise occlusion in the image during the query, resulting in inaccurate retrieval results. Summary of the invention

[0004] The present invention provides a picture retrieval method to achieve accurate acquisition of retrieval pictures.

[0005] According to a first aspect of the present invention, there is provided a method for image retrieval, comprising: mapping a query object to a query data point in an embedding space, wherein the query object comprises an image or an image description text;

[0006] Converting the query data point into a query vector in the embedding space, and converting the sample data point in the retrieval database into a sample vector, wherein the retrieval database includes the sample data point corresponding to each sample image;

[0007] Determine a target sample vector corresponding to the query vector, and obtain a target sample image corresponding to the target sample vector;

[0008] The target sample image is used as the image retrieval result of the query object.

[0009] Optionally, map the query object to a query data point in the embedding space, including:

[0010] Acquire an associative memory model, wherein the associative memory model includes a generation network, an embedding function, and a similarity function;

[0011] The query object is mapped to a query data point in an embedding space through the associative memory model.

[0012] Optionally, mapping the query object to a query data point in an embedding space through the associative memory model includes:

[0013] When the query object is a picture, the picture is identified to obtain picture description text, and the generation network is used to map the picture description text to the query data point in the embedding space;

[0014] When the query object is a picture description text, the generative network is directly used to map the picture description text to the query data point in the embedding space.

[0015] Optionally, converting the query data point into a query vector in the embedding space, and converting the sample data point in the retrieval database into a sample vector, comprises:

[0016] converting the query data point into the query vector using the embedding function in the embedding space;

[0017] Determining a conversion mode for the search database, wherein the conversion mode includes parallel conversion or sequential conversion;

[0018] The embedding function is used in the embedding space to convert the sample data points in the retrieval database into the sample vector based on the conversion pattern.

[0019] According to another aspect of the present invention, there is provided an image retrieval device, comprising: a query object mapping module, for mapping a query object to a query data point in an embedding space, wherein the query object comprises an image or an image description text;

[0020] A data point conversion module, used to convert the query data point into a query vector in the embedding space, and convert the sample data point in the retrieval database into a sample vector, wherein the retrieval database includes the sample data point corresponding to each sample image;

[0021] A target sample image acquisition module, used to determine a target sample vector corresponding to the query vector, and acquire a target sample image corresponding to the target sample vector;

[0022] The image retrieval result determination module is used to use the target sample image as the image retrieval result of the query object.

[0023] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0024] at least one processor; and

[0025] a memory communicatively connected to the at least one processor; wherein,

[0026] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the method described in any embodiment of the present invention.

[0027] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method described in any embodiment of the present invention when executed.

[0028] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, and when the computer program is executed by a processor, the method described in any embodiment of the present invention is implemented.

[0029] The beneficial technical effect of the present invention is that sample images are saved in the form of sample data points in the retrieval database, thereby reducing the storage pressure of the database, and when performing image retrieval, whether the user inputs an image or image description text, it is converted into a meaningful query data point form, and the retrieval query is performed in the form of data points, thereby avoiding the influence of factors such as noise in the image, which may cause inaccurate image retrieval results.

[0030] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 is a flowchart of a picture retrieval method provided according to the first embodiment of the present invention;

[0033] Figure 2 is a flowchart of a picture retrieval method provided according to Embodiment 2 of the present invention;

[0034] Figure 3 is a structural schematic diagram of an image retrieval device provided according to Embodiment 3 of the present invention;

[0035] Figure 4 It is a schematic diagram of the structure of an electronic device provided by Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] Embodiment 1

[0039] Figure 1 A flowchart of a picture retrieval method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of retrieving pictures. The method can be executed by a picture retrieval device, which can be implemented in the form of hardware and / or software. Figure 1 As shown, the method includes:

[0040] Step S101, mapping the query object to a query data point in the embedding space.

[0041] Optionally, mapping the query object to a query data point in an embedding space includes: obtaining an associative memory model, wherein the associative memory model includes a generative network, an embedding function, and a similarity function; and mapping the query object to a query data point in the embedding space through the associative memory model.

[0042] Optionally, the query object is mapped to a query data point in the embedding space through an associative memory model, including: when the query object is a picture, the picture is identified to obtain picture description text, and a generative network is used to map the picture description text to a query data point in the embedding space; when the query object is picture description text, a generative network is directly used to map the picture description text to a query data point in the embedding space.

[0043] Specifically, in this embodiment, an associative memory model can be used to map query objects, wherein the associative memory model mainly includes a generation network, an embedding function and a similarity function, wherein the generation network is mainly used to map text to data points in the embedding space, the embedding function is mainly used to convert data points into vectors, and the similarity function is mainly used to calculate the similarity between vectors. Of course, this embodiment is only an example and does not limit the specific structure contained in the associative memory model.

[0044] Among them, the generative network in the associative memory model can map the image description text to the data points in the embedding space. The image description text is highly random and uncertain. The generative network can transform the irregular image description text into meaningful data points in the embedding space by learning a large number of data sample patterns. This lays the foundation for the subsequent models to reserve rich and diverse data, broadens the model's ability to cope with different data forms, increases the richness of the data, and prevents the model from being limited to the existing data, thereby improving its generalization.

[0045] In a specific implementation, when the query object is a picture, it is necessary to first identify the picture to obtain the picture description text. Specifically, image recognition can be used to obtain valid information in the picture, and the valid information is combined to generate the picture description text. Therefore, the picture description text contains semantic information related to the picture, and a generative network is used to map the picture description text to a query data point in the embedding space. The generative network can convert the picture description text into meaningful data points in the embedding space by learning a large number of data sample patterns. Due to factors such as picture noise occlusion, the conversion from text to data points cannot be achieved, so that only meaningful information related to the picture is included in the data point. In addition, when the query object is a picture description text, the generative network is directly used to map the text to obtain the query data point. Therefore, relative to the picture, when the query object is a picture description text, there is no need to perform picture recognition.

[0046] Step S102, converting the query data points into query vectors in the embedding space, and converting the sample data points in the retrieval database into sample vectors.

[0047] Optionally, query data points are converted into query vectors in the embedding space, and sample data points in the retrieval database are converted into sample vectors, including: using an embedding function in the embedding space to convert the query data points into query vectors; determining a conversion mode for the retrieval database, wherein the conversion mode includes parallel conversion or sequential conversion; and using an embedding function in the embedding space to convert sample data points in the retrieval database into sample vectors based on the conversion mode.

[0048] Specifically, in this embodiment, the sample images are converted into image description texts, and all of them are mapped into sample data points in the embedding space using the associative memory model. The sample images can be saved in the cloud server, and the relationship between each sample image and the sample data point is established, such as marking each sample data point in the form of a number, etc. Of course, this embodiment is only an example, and does not limit the association method between each sample image and the sample data point. In addition, in this embodiment, the sample data points corresponding to each sample image are saved in the local retrieval database. Compared with image storage, the storage method of sample points will significantly reduce the storage pressure of the database, thereby improving the local memory efficiency, and by converting the image into text through semantic recognition, and mapping the semantically recognized text into meaningful data points in the embedding space, the semantic information of the image is stored in the form of data points. In addition, in the embedding space mapping process, since the text is converted, the noise and occlusion in the image can be eliminated, so that only the sample data points corresponding to the effective semantic content in the sample image are saved in the retrieval database, which improves the accuracy of the data points in the data.

[0049] It should be noted that the embedding function in this embodiment can accurately translate the data points in the embedding space into a low-dimensional, compact vector representation in the embedding space. By converting to a vector representation, on the one hand, the data dimension is reduced, and the storage and computing pressure is reduced; on the other hand, data points with similar semantics are closer after the embedding conversion, and data points with different semantics are farther away from each other, which facilitates subsequent rapid retrieval and distinction, and provides a structured basis for similarity judgment.

[0050] In a specific implementation, after obtaining the query data point, the embedding function will be used in the embedding space to further convert the query data point into a query vector, and the sample data points in the retrieval database will also be converted into sample vectors. Since there are many sample data points in the retrieval database, a parallel conversion mode can be used to convert a specified number of sample data points when the computing resources are sufficient, for example, 30 groups of sample data points are used for parallel conversion. Of course, this embodiment is only an example, and the number of sample data points for parallel conversion is not specifically limited; when the computing resources are insufficient, in order to ensure the accuracy of the conversion, the sample data points in the retrieval database can be converted in a sequential conversion manner, which is not limited in this embodiment. The conversion mode can be pre-configured or determined according to the user mode selection instruction during the conversion.

[0051] Step S103, determining a target sample vector corresponding to the query vector, and obtaining a target sample image corresponding to the target sample vector.

[0052] Optionally, determining the target sample vector corresponding to the query vector includes: calculating a similarity value between the query vector and each sample vector by using a similarity function; and taking a sample vector whose similarity value meets a preset threshold as the target sample vector.

[0053] Among them, the similarity score between data points in the embedded space can be calculated through the similarity function in the associative memory model. That is, after obtaining the data point vector in the embedded space, the distance, angle and other features between each vector are quickly and accurately compared to quantify their similarity and give a similarity score. In the image retrieval scenario, a high score means that the semantics of the two images are very similar, and a low score means the opposite. This allows the model to efficiently screen out data that matches the query target and speed up the retrieval process.

[0054] Specifically, in this embodiment, after obtaining the similarity value between the query vector and each sample vector, the sample vector whose similarity value meets the preset threshold value will be used as the target sample vector, wherein the preset threshold value can be determined by the user according to the retrieval accuracy, and the specific value thereof is not limited in this embodiment, for example, it can be 99%, and the sample vector whose similarity value is greater than 99% will be used as the target sample vector, and the number of the obtained target sample vectors is at least one. Of course, this embodiment is only an example, and the specific number of the preset threshold value is not limited.

[0055] It should be noted that in this embodiment, after obtaining the target sample vector, since the correspondence between each sample vector and the sample image in the cloud server is pre-established, the number of the target sample vector can be obtained, and the corresponding sample image can be retrieved from the cloud server according to the number. Therefore, there is no need to save the image locally. Only when the target sample vector is determined, the corresponding target sample image is retrieved from the cloud server and saved locally, which significantly improves the efficiency of local memory and reduces the storage pressure of the local model.

[0056] Step S104: taking the target sample image as the image retrieval result of the query object.

[0057] Optionally, taking the target sample image as the image retrieval result of the query object includes: determining whether the number of target sample images is one, if so, directly taking the target sample image as the image retrieval result; otherwise, obtaining a pre-set user filtering rule, and determining the target sample image that matches the user filtering rule from the target sample images, and taking the filtered target sample image as the image retrieval result.

[0058] Among them, in this embodiment, when the target sample pictures are obtained, different methods are used to obtain the image retrieval results according to the different numbers of target sample pictures. When the number of target sample pictures is one, the only target sample picture obtained can be directly used as the image retrieval result. Another case is when the number of target sample pictures is multiple, in order to improve the retrieval accuracy while meeting the personalized needs of users, the target sample pictures can be further screened using pre-set user screening rules, and the target sample pictures matching the user screening rules are used as the image retrieval results. Among them, the user screening rules include: screening the top three pictures with the highest similarity according to the principle of precision priority, using all the obtained target sample pictures as the image retrieval results according to the principle of comprehensive screening, or using the target sample pictures containing building logos as the image retrieval results according to the principle of specified element screening. Of course, this embodiment is only an example, and does not limit the specific content of the user screening rules.

[0059] In a specific implementation, when the user filtering rule is "filtering the image with the highest similarity according to the principle of precision priority", it indicates that the user currently wants to retrieve an image that is most similar to the query object, so multiple target sample images are sorted, and the target sample image with the highest similarity in the sequence is used as the image retrieval result, and is displayed on the front-end visualization interface; when the user filtering rule is "using the comprehensive filtering principle to use all the acquired target sample images as the image retrieval result", it indicates that the user currently wants to retrieve as many images as possible that are similar to the query object, so all the acquired target sample images are used as the image retrieval result, for example, 50 sample images, and the 50 acquired sample images are used as the image retrieval result, and are displayed on the front-end visualization interface; when the user filtering rule is "using the index to select the image", it indicates that the user currently wants to retrieve as many images as possible that are similar to the query object, so all the acquired target sample images are used as the image retrieval result, for example, 50 sample images, and the 50 sample images are used as the image retrieval result, and are displayed on the front-end visualization interface; When the user inputs the target sample pictures containing building logos as the picture retrieval results according to the element screening principle, it indicates that the user wants to further screen the target sample pictures according to the retrieval requirements. For example, the query object initially input by the user is the picture description text "Please search for city-style pictures". After obtaining 200 target sample pictures according to the user's retrieval requirements, if the user has a new retrieval requirement after seeing the sample pictures and wants to view pictures containing only building logos, there is no need to repeatedly input the query object for retrieval. Instead, it is only necessary to filter from the 200 target sample pictures according to the set user screening rules based on the current retrieval results, so as to obtain 50 city-style pictures containing building logos, thereby avoiding the problem of resource occupation and low retrieval efficiency caused by repeatedly inputting the query object. Of course, the present implementation is only an example and does not limit the specific screening method.

[0060] In the implementation mode of the present application, sample images are saved in the form of sample data points in the retrieval database, thereby reducing the storage pressure of the database, and when performing image retrieval, whether the user inputs an image or a description of the image, it is converted into a meaningful query data point form, and the retrieval query is performed in the form of data points, thereby avoiding the influence of factors such as noise in the image, which may lead to inaccurate image retrieval results.

[0061] Embodiment 2

[0062] Figure 2 This is a flowchart of an image retrieval method provided in the second embodiment of the present invention. This embodiment, based on the above embodiment, takes the target sample image as the image retrieval result of the query object, and further includes: testing the image retrieval result, and adjusting the parameters of the association memory model when it is determined that the detection fails. Figure 2 As shown, the method includes:

[0063] Step S201, mapping the query object to a query data point in the embedding space.

[0064] Optionally, mapping the query object to a query data point in an embedding space includes: obtaining an associative memory model, wherein the associative memory model includes a generative network, an embedding function, and a similarity function; and mapping the query object to a query data point in the embedding space through the associative memory model.

[0065] Optionally, the query object is mapped to a query data point in the embedding space through an associative memory model, including: when the query object is a picture, the picture is identified to obtain picture description text, and a generative network is used to map the picture description text to a query data point in the embedding space; when the query object is picture description text, a generative network is directly used to map the picture description text to a query data point in the embedding space.

[0066] Step S202, converting the query data points into query vectors in the embedding space, and converting the sample data points in the retrieval database into sample vectors.

[0067] Optionally, query data points are converted into query vectors in the embedding space, and sample data points in the retrieval database are converted into sample vectors, including: using an embedding function in the embedding space to convert the query data points into query vectors; determining a conversion mode for the retrieval database, wherein the conversion mode includes parallel conversion or sequential conversion; and using an embedding function in the embedding space to convert sample data points in the retrieval database into sample vectors based on the conversion mode.

[0068] Step S203, determining a target sample vector corresponding to the query vector, and obtaining a target sample image corresponding to the target sample vector.

[0069] Optionally, determining the target sample vector corresponding to the query vector includes: calculating a similarity value between the query vector and each sample vector by using a similarity function; and taking a sample vector whose similarity value meets a preset threshold as the target sample vector.

[0070] Step S204: taking the target sample image as the image retrieval result of the query object.

[0071] Optionally, taking the target sample image as the image retrieval result of the query object includes: determining whether the number of target sample images is one, if so, directly taking the target sample image as the image retrieval result; otherwise, obtaining a pre-set user filtering rule, and determining the target sample image that matches the user filtering rule from the target sample images, and taking the filtered target sample image as the image retrieval result.

[0072] Step S205, detecting the image retrieval result, and adjusting the parameters of the associative memory model when it is determined that the detection fails.

[0073] Specifically, in this embodiment, after obtaining the image retrieval results, the image retrieval results will also be tested, that is, after obtaining the image retrieval results, the user's score for the image retrieval results can be received. When the score result is relatively low, it means that the obtained image retrieval results are not matched with the query object in terms of the degree of match, and are not the results that the user wants to query. In this case, it is determined that the image retrieval results are wrong, that is, the detection fails.

[0074] It should be noted that the reason for the above detection failure may be an error in the associative memory model or an error in the hardware device itself. The specific reason for the detection failure is not limited in this implementation. In addition, an alarm message will be generated in the event of a detection failure to prompt the operation and maintenance personnel to adjust the hardware device or the associative memory model parameters in a timely manner, thereby ensuring the accuracy of subsequent image retrieval.

[0075] It is worth mentioning that in order to save operating resources, in this implementation, a periodic timing method can be used for detection, that is, a detection request will be sent to the operation and maintenance personnel every other day. When the operation and maintenance personnel receive the verification request, they can accept or reject it. When it is confirmed that it is accepted, the current image detection result is scored to obtain the detection result, so that when an image retrieval error occurs, regular maintenance of the algorithm or hardware equipment can be achieved.

[0076] In the implementation mode of the present application, sample images are saved in the form of sample data points in the retrieval database, thereby reducing the storage pressure of the database, and when performing image retrieval, whether the user inputs an image or a description of the image, it is converted into a meaningful query data point form, and the retrieval query is performed in the form of data points, thereby avoiding the influence of factors such as noise in the image, which may lead to inaccurate image retrieval results.

[0077] Embodiment 3

[0078] Figure 3 This is a schematic diagram of the structure of an image retrieval device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: a query object mapping module 310, a data point conversion module 320, a target sample image acquisition module 330 and an image retrieval result determination module 340.

[0079] A query object mapping module 310, for mapping a query object to a query data point in an embedding space, wherein the query object includes a picture or a picture description text;

[0080] A data point conversion module 320, configured to convert a query data point into a query vector in the embedding space, and convert a sample data point in a search database into a sample vector, wherein the search database includes sample data points corresponding to each sample image;

[0081] A target sample image acquisition module 330 is used to determine a target sample vector corresponding to a query vector, and to acquire a target sample image corresponding to the target sample vector;

[0082] The image retrieval result determination module 340 is used to determine the image retrieval result using the target sample image as the query object.

[0083] Optionally, a query object mapping module is used to obtain an associative memory model, wherein the associative memory model includes a generative network, an embedding function, and a similarity function;

[0084] The query object is mapped to the query data point in the embedding space through the associative memory model.

[0085] Optionally, the query object mapping module is further used to, when the query object is a picture, identify the picture to obtain picture description text, and use a generative network to map the picture description text to a query data point in the embedding space;

[0086] When the query object is a picture description text, the generative network is directly used to map the picture description text to the query data point in the embedding space.

[0087] Optionally, a data point conversion module, for converting a query data point into a query vector using an embedding function in an embedding space;

[0088] Determining a conversion mode for searching a database, wherein the conversion mode includes parallel conversion or sequential conversion;

[0089] In the embedding space, an embedding function is used to convert sample data points in the retrieval database into sample vectors based on the conversion pattern.

[0090] Optionally, the target sample image acquisition module is further used to calculate the similarity value between the query vector and each sample vector through a similarity function;

[0091] The sample vector whose similarity value meets the preset threshold is taken as the target sample vector.

[0092] Optionally, the image retrieval result determination module is used to determine whether the number of target sample images is one, and if so, directly use the target sample image as the image retrieval result.

[0093] Otherwise, a preset user screening rule is obtained, and target sample images matching the user screening rule are determined from the target sample images, and the screened target sample images are used as image retrieval results.

[0094] Optionally, the device further includes a detection module, which is used to detect the image retrieval result;

[0095] When it is determined that the detection fails, the parameters of the associative memory model are adjusted.

[0096] The image retrieval device provided in the embodiment of the present invention can execute the image retrieval method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0097] Embodiment 4

[0098] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0099] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0100] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0101] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the image retrieval method.

[0102] In some embodiments, the method for image retrieval may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the image retrieval method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the image retrieval method in any other appropriate manner (e.g., by means of firmware).

[0103] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0105] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0107] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0108] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0109] Embodiment 5

[0110] An embodiment of the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the image retrieval method provided in any embodiment of the present application.

[0111] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0112] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0113] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A picture retrieval method, characterized in that: include: Mapping a query object to a query data point in an embedding space, wherein the query object includes a picture or a picture description text; Converting the query data point into a query vector in the embedding space, and converting the sample data point in the retrieval database into a sample vector, wherein the retrieval database includes the sample data point corresponding to each sample image; Determine a target sample vector corresponding to the query vector, and obtain a target sample image corresponding to the target sample vector; The target sample image is used as the image retrieval result of the query object.

2. The method according to claim 1, characterized in that: Mapping the query object to a query data point in the embedding space includes: Acquire an associative memory model, wherein the associative memory model includes a generation network, an embedding function, and a similarity function; The query object is mapped to a query data point in an embedding space through the associative memory model.

3. The method according to claim 2, characterized in that Mapping the query object to a query data point in an embedding space by using the associative memory model includes: When the query object is a picture, the picture is identified to obtain picture description text, and the generation network is used to map the picture description text to the query data point in the embedding space; When the query object is a picture description text, the generative network is directly used to map the picture description text to the query data point in the embedding space.

4. The method according to claim 2, characterized in that: The step of converting the query data point into a query vector in the embedding space and converting the sample data point in the retrieval database into a sample vector comprises: converting the query data point into the query vector using the embedding function in the embedding space; Determining a conversion mode for the search database, wherein the conversion mode includes parallel conversion or sequential conversion; The embedding function is used in the embedding space to convert the sample data points in the retrieval database into the sample vector based on the conversion mode.

5. The method according to claim 2, characterized in that: The determining of the target sample vector corresponding to the query vector includes: Calculating similarity values ​​between the query vector and each sample vector by using the similarity function; The sample vector whose similarity value meets a preset threshold is used as the target sample vector.

6. The method according to claim 1, characterized in that The image retrieval result using the target sample image as the query object includes: When determining whether the number of the target sample image is one, if so, directly use the target sample image as the image retrieval result. Otherwise, a preset user screening rule is obtained, and target sample pictures matching the user screening rule are determined from the target sample pictures, and the screened target sample pictures are used as the picture retrieval results.

7. The method according to claim 2, characterized in that After taking the target sample image as the image retrieval result of the query object, the method further includes: Testing the image retrieval result; When it is determined that the detection fails, parameters of the associative memory model are adjusted.

8. A picture retrieval device, characterized in that: The device comprises: A query object mapping module, used to map a query object to a query data point in an embedding space, wherein the query object includes a picture or a picture description text; A data point conversion module, used to convert the query data point into a query vector in the embedding space, and convert the sample data point in the retrieval database into a sample vector, wherein the retrieval database includes the sample data point corresponding to each sample image; A target sample image acquisition module, used to determine a target sample vector corresponding to the query vector, and acquire a target sample image corresponding to the target sample vector; The image retrieval result determination module is used to use the target sample image as the image retrieval result of the query object.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 7 when executed.

11. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images