Image retrieval method and device, electronic equipment, storage medium and product

Abstract

The application provides an image retrieval method and device, electronic equipment, storage medium and product, and belongs to the technical field of image retrieval. The method comprises the following steps: receiving an image retrieval instruction, wherein the image retrieval instruction carries a first image; performing subject segmentation on the first image to obtain a plurality of metadata of the first image, wherein the plurality of metadata comprises a plurality of subjects included in the first image and attribute information of the plurality of subjects; determining a retrieval feature of the first image based on the plurality of metadata and a target retrieval option, wherein the retrieval feature is used for similarity matching with other images, and the target retrieval option is used for determining a retrieval feature from the plurality of metadata; and retrieving a second image matched with the first image from an image library based on the retrieval feature. The application can improve the accuracy of image retrieval.

Description

Technical Field

[0001] This application relates to the field of image retrieval technology, and in particular to an image retrieval method, apparatus, electronic device, storage medium and product. Background Technology

[0002] Currently, image retrieval scenarios using image search are becoming increasingly common; for example, users can search for similar items in shopping applications using images that include the items. In related technologies, the process by which an electronic device retrieves a similar second image based on a first image input by the user can be as follows: extracting global features of the first image; and based on the global features of the first image, retrieving a second image from an image library whose global features are more similar to the global features of the first image than a preset similarity threshold.

[0003] Related technologies perform image retrieval based on global features. However, relying on global features may not retrieve the images that the user wants. For example, a first image may contain a first object and a second object, while a second image may contain the first object. If the user is interested in the first object in the first image, that is, wants to search for images containing the first object, the second image will not be retrieved based on the global features of the first image, resulting in low accuracy of image retrieval. Summary of the Invention

[0004] This application provides an image retrieval method, apparatus, electronic device, storage medium, and product, which can improve the accuracy of image retrieval. The technical solution is as follows:

[0005] On the one hand, an image retrieval method is provided, the method comprising:

[0006] Receive an image retrieval instruction, wherein the image retrieval instruction carries a first image;

[0007] The first image is segmented to obtain multiple metadata of the first image, including multiple subjects included in the first image and attribute information of the multiple subjects;

[0008] Based on the plurality of metadata and target retrieval options, retrieval features of the first image are determined. These retrieval features are used for similarity matching with other images. The target retrieval options are used to determine retrieval features from the plurality of metadata.

[0009] Based on the search features, a second image matching the first image is retrieved from the image library.

[0010] On the other hand, an image retrieval device is provided, the device comprising:

[0011] A receiving module is configured to receive an image retrieval instruction, wherein the image retrieval instruction carries a first image;

[0012] The segmentation module is used to perform subject segmentation on the first image to obtain multiple metadata of the first image, wherein the multiple metadata includes multiple subjects included in the first image and attribute information of the multiple subjects;

[0013] The determining module is used to determine the retrieval features of the first image based on the plurality of metadata and target retrieval options, wherein the retrieval features are used for similarity matching with other images, and the target retrieval options are used to determine the retrieval features from the plurality of metadata;

[0014] The retrieval module is used to retrieve a second image that matches the first image from the image library based on the retrieval features.

[0015] On the other hand, an electronic device is provided, comprising one or more processors and one or more memories, wherein at least one piece of program code is stored in the one or more memories, and the at least one piece of program code is loaded and executed by the one or more processors to implement the image retrieval method described in any of the above implementations.

[0016] On the other hand, a computer-readable storage medium is provided, wherein at least one piece of program code is stored in the computer-readable storage medium, the at least one piece of program code being loaded and executed by a processor to implement the image retrieval method described in any of the above implementations.

[0017] On the other hand, a computer program product is provided, the computer program product storing at least one piece of program code, the at least one piece of program code being executed by a processor to implement the image retrieval method as described in any of the above implementations.

[0018] In this embodiment, on the one hand, by performing subject segmentation on the first image, multiple metadata of the first image are obtained, including multiple subjects included in the first image and attribute information of the subjects. Therefore, this application can obtain more granular metadata. On the other hand, by using target retrieval options, retrieval features can be determined from the multiple metadata, thus enabling image retrieval to be more targeted. Based on the above two points, it can be seen that this application can perform image retrieval in a more granular and targeted manner, thereby improving the accuracy of image retrieval. Attached Figure Description

[0019] Figure 1 A schematic diagram illustrating the implementation environment of an image retrieval method according to an exemplary embodiment of this application is shown;

[0020] Figure 2 A flowchart illustrating an exemplary embodiment of the image retrieval method of this application is shown;

[0021] Figure 3 A flowchart illustrating an exemplary embodiment of the image retrieval method of this application is shown;

[0022] Figure 4 A flowchart illustrating an exemplary embodiment of the image retrieval method of this application is shown;

[0023] Figure 5 A flowchart illustrating an exemplary embodiment of the image retrieval method of this application is shown;

[0024] Figure 6 A flowchart illustrating an exemplary embodiment of the image retrieval method of this application is shown;

[0025] Figure 7 A schematic diagram illustrating image segmentation is shown in an exemplary embodiment of this application;

[0026] Figure 8 A schematic diagram illustrating the selection of multiple second subjects is shown in an exemplary embodiment of this application;

[0027] Figure 9 A schematic diagram illustrating the adjustment of the positional relationship of multiple second entities is shown in an exemplary embodiment of this application;

[0028] Figure 10 A flowchart illustrating an exemplary embodiment of the image retrieval method of this application is shown;

[0029] Figure 11 A flowchart illustrating an exemplary embodiment of the image retrieval method of this application is shown;

[0030] Figure 12 A flowchart illustrating an exemplary embodiment of the image retrieval method of this application is shown;

[0031] Figure 13 A flowchart illustrating an exemplary embodiment of the image retrieval method of this application is shown;

[0032] Figure 14 A flowchart illustrating an exemplary embodiment of the image retrieval method of this application is shown;

[0033] Figure 15 A block diagram of an image retrieval apparatus illustrated in an exemplary embodiment of this application is shown;

[0034] Figure 16 A block diagram of a terminal illustrated in an exemplary embodiment of this application is shown;

[0035] Figure 17 A block diagram of a server illustrated in an exemplary embodiment of this application is shown. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0037] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0038] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the images involved in this application (including but not limited to the first image, the second image, the third image, and the fourth image) were obtained with full authorization.

[0039] Please refer to Figure 1 This illustration shows a schematic diagram of an implementation environment for an image retrieval method according to an exemplary embodiment of this application. The implementation environment includes an electronic device, which can be configured as a terminal 101, a server 102, or both, enabling interaction between the terminal 101 and the server 102 to achieve the image retrieval function. Figure 1 The following description uses an electronic device configured as terminal 101 and server 102 as an example. Terminal 101 has an image retrieval application installed. Terminal 101 uses this application to communicate with server 102, performing functions such as data transmission and information interaction. Terminal 101 requests image retrieval from server 102's image library; alternatively, terminal 101 stores its own image library, allowing it to perform offline image retrieval. The image retrieval application can be a shopping application, a photo album application, or a test question search application, etc.

[0040] In some embodiments, terminal 101 may be a smartphone, tablet, laptop, desktop computer, or vehicle terminal, but is not limited thereto. Server 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0041] This application provides several image retrieval scenarios in its embodiments:

[0042] The first scenario: The image retrieval application is a shopping app. When a user sees a first image containing a first item the user likes, but the first image may also contain other items, the user can trigger the electronic device to retrieve a second image containing that first item from the shopping app's image library. Alternatively, when a user sees a first image containing the shape of an item the user likes, the user can trigger the electronic device to retrieve a second image containing that item's shape from the shopping app's image library.

[0043] The second scenario: Image retrieval application is a test question search application; when a user is doing a geometry test question, the user can take a first image containing the geometry test question, and then trigger the electronic device to retrieve a second image containing the geometry test question from the image library (test question library) based on the first image.

[0044] The third scenario: The image retrieval application is a photo album application; when a user sees the first image in the photo album application, the first image is a selfie of the user, and the user is wearing a hat in the first image. If the user wants to search whether the photo album application also includes an image of this hat, the user can trigger the electronic device to search the image library (photo album) for a second image containing the hat based on the first image.

[0045] In addition, the image retrieval method provided in the application embodiment can also be applied to the scenario of similar image judgment; correspondingly, the image retrieval method can be used to determine whether the third image and the fourth image are similar images.

[0046] Please refer to Figure 2 This document illustrates a flowchart of an image retrieval method according to an exemplary embodiment of this application. The execution entity in this embodiment is an electronic device. See also... Figure 2 The method includes:

[0047] Step 201: Receive an image retrieval instruction, which carries the first image.

[0048] When an electronic device is configured as a server, an image retrieval application is installed on the terminal. The user can trigger a first image to the terminal using this application. The terminal then sends an image retrieval command to the server, which receives the command. This image retrieval application can be a photo album application, a shopping application, or a test question search application. For example, please refer to... Figure 3 The user triggers a first image to the electronic device. If the electronic device is configured as a terminal, the terminal receives the image retrieval command triggered by the user.

[0049] Step 202: Perform subject segmentation on the first image to obtain multiple metadata of the first image. The multiple metadata includes multiple subjects included in the first image and attribute information of the multiple subjects.

[0050] The first image is segmented into its main subject using a general-purpose Segment Anything Model (SAM). Unlike previous image segmentation models that could only handle specific image types, the SAM model can segment all image types; therefore, using the SAM model for main subject segmentation of the first image improves its versatility. For example, please refer to [link to relevant documentation]. Figure 3 The electronic device performs subject segmentation on the first image using the SAM model.

[0051] For any subject, the subject's attribute information may include at least one of the following: the subject's shape, the subject's color, the number of subjects, the subject's position in the first image, and other attribute information. If the subject is a user, the other attribute information may include at least one of the user's actions (such as raising a hand or kicking a leg) and expressions (such as laughing or crying). If the subject is an object, the other attribute information may include the object's state; for example, if the subject is a water cup, other attributes of the water cup may include whether the water cup has a lid, whether the water cup is tilted, etc.

[0052] In some embodiments, when the first image includes multiple subjects, the metadata of the first image includes the multiple subjects included in the first image and the attribute information of the multiple subjects. When the first image includes only one subject, the metadata of the first image includes the one subject included in the first image and the attribute information of that subject. In this embodiment, subject segmentation of the first image using the SAM model enables fine-grained subject segmentation and the establishment of multiple metadata used for search.

[0053] Step 203: Based on multiple metadata and target retrieval options, determine the retrieval features of the first image. The retrieval features are used for similarity matching with other images, and the target retrieval options are used to determine the retrieval features from multiple metadata.

[0054] The target retrieval options can be any one of the following: single-subject option, multi-subject option, and attribute option. The multi-subject option can be further divided into a first multi-subject option and a second multi-subject option. The first multi-subject option is used to select multiple subjects from the segmented multiple subjects, while the second multi-subject option is used to adjust the positional relationship of the selected subjects after selection. The attribute options are further divided into shape options, color options, and other attribute options.

[0055] Step 204: Based on the retrieval features, retrieve a second image from the image library that matches the first image.

[0056] In some embodiments, if the retrieval feature includes a first subject in the first image, then the retrieved second image will also include the first subject. In other embodiments, if the retrieval feature includes multiple subjects in the first image, then the retrieved second image will also include all of those subjects. In still other embodiments, if the retrieval feature includes attribute information of a third subject in the first image, then the retrieved second image will also include the attribute information of the third subject; for example, if the retrieval feature includes the shape of the third subject, then the retrieved second image will also include the shape of the third subject; as another example, if the retrieval feature includes the color of the third subject, then the retrieved second image will also include the color of the third subject; as yet another example, if the retrieval feature includes other attributes of the third subject (e.g., a user's laughing expression), then the retrieved second image will also include other attributes of the third subject (e.g., a user's laughing expression).

[0057] In this embodiment, on the one hand, by performing subject segmentation on the first image, multiple metadata of the first image are obtained, including multiple subjects included in the first image and attribute information of the subjects. Therefore, this application can obtain more granular metadata. On the other hand, by using target retrieval options, retrieval features can be determined from the multiple metadata, thus enabling image retrieval to be more targeted. Based on the above two points, it can be seen that this application can perform image retrieval in a more granular and targeted manner, thereby improving the accuracy of image retrieval.

[0058] Please refer to Figure 4 The diagram illustrates a flowchart of an image retrieval method according to an exemplary embodiment of this application. See also... Figure 4 The method includes:

[0059] Step 401: The electronic device receives an image retrieval instruction, which carries the first image.

[0060] In some embodiments, this step is the same as step 201, and will not be described again here.

[0061] Step 402: The electronic device performs subject segmentation on the first image to obtain multiple metadata of the first image. The multiple metadata includes multiple subjects included in the first image and attribute information of the multiple subjects.

[0062] In some embodiments, this step is the same as step 202, and will not be described again here.

[0063] Step 403: The electronic device displays multiple search options.

[0064] The electronic device displays an editing interface, which includes multiple search options and multiple subjects. The search options include single-subject options, multi-subject options, and multiple attribute options. The multi-subject options include a first multi-subject option and a second multi-subject option. The first multi-subject option is used to select multiple subjects from the segmented multiple subjects, while the second multi-subject option is used to adjust the positional relationships of the selected subjects. The attribute options are divided into shape options, color options, and other attribute options. In addition, the electronic device can perform full-image retrieval, which refers to image retrieval based on the multiple subjects and their attribute information included in the first image. Correspondingly, the editing interface includes multiple search options such as full-image retrieval, single-subject option, first multi-subject option, second multi-subject option, shape option, color option, and other attribute options.

[0065] For example, please continue to refer to Figure 3 The electronic device displays multiple search options, allowing the user to select a search method. Different search options correspond to different search methods. In this embodiment, displaying multiple search options and allowing the user to choose a search method enhances the purposefulness and flexibility of the search. Furthermore, users can perform searches in various ways based on full-image search, single-subject options, first multi-subject options, second multi-subject options, shape options, color options, and other attribute options, thereby achieving more granular image retrieval, making image retrieval more targeted, and thus improving the efficiency of image retrieval.

[0066] In some embodiments, the electronic device also supports image retrieval via voice or direct image input. Accordingly, the voice retrieval process can be as follows: the electronic device receives an image retrieval command carrying a voice signal, converts the voice signal into text information, and retrieves a second image matching the text information from the image library. The image retrieval process via direct image input can be as follows: the electronic device receives an image retrieval command carrying a first image, and retrieves a second image similar to the first image from the image library based on the first image. In this embodiment, not only is image retrieval based on finer-grained subject and / or subject attribute information supported, but voice or direct image input is also supported, improving the comprehensiveness of image retrieval.

[0067] When an electronic device is configured as a server, the steps for the electronic device to display multiple search options can be as follows: the server sends interface data of the editing interface to the terminal, which includes multiple search options; the terminal receives the interface data of the editing interface, renders the editing interface based on the interface data, and the editing interface displays multiple search options.

[0068] Step 404: When the single subject option among multiple search options is detected to be selected, the electronic device determines the first subject from multiple metadata and uses the first subject as the search feature of the first image.

[0069] The electronic device displays an editing interface with multiple search options and subjects. Users can select search options and subjects based on this interface. In some embodiments, the user selects a search option first and then a subject. This process can be as follows: the user can select a search option by clicking; for example, the user clicks on a single subject option to select it. The electronic device detects that the single subject option is selected, and then the user clicks on a first subject, at which point the electronic device identifies the first subject as selected. For example, if the user wants to search for a cup in the first image, the device can simply click on the cup.

[0070] In other embodiments, the user first selects a subject and then selects a search option. This process can be as follows: the user clicks on a first subject, and then clicks on a single subject option. At this time, the electronic device determines that the single subject option among multiple search options is selected and identifies the selected first subject. It should be noted that the first subject includes not only the subject itself, but also the subject's attribute information; for example, if the first subject is a water cup, and the water cup's attribute information includes color (green), then the search features of the first image include water cup and green, that is, the search feature of the first image is a green water cup. In the embodiments of this application, the feature category of the search features of the first image is a single subject feature.

[0071] When the electronic device is configured as a server, the terminal displays an editing interface and checks that the single subject option among the multiple search options in the editing interface is selected. It then determines the first subject from multiple metadata, sends the subject identifier of the first subject to the server, receives the subject identifier of the first subject, and uses the first subject as the search feature of the first image based on the subject identifier of the first subject.

[0072] Step 405: The electronic device retrieves a second image that matches the first image from the image library based on the retrieval features.

[0073] The image library includes multiple second images. For any second image, the electronic device extracts image features, which include the subject, color, and other attribute information of the second image, excluding color and shape. Based on the retrieval features and the image features of the second image, the electronic device determines the similarity between the first image and the second image. For example, the similarity between the retrieval features and the image features of the second image is the same as the similarity between the first image and the second image. Based on this method, the similarity between the first image and multiple second images can be determined. Based on the similarity between the first image and multiple second images, the electronic device determines a second image from the image library whose similarity exceeds a first preset similarity. The determined second image is the second image that matches the first image. The number of second images can be one or multiple.

[0074] When an electronic device is configured as a server, after the server determines the second image, it sends the second image to the terminal. The terminal receives the second image and displays it, thereby presenting the image retrieval results to the user.

[0075] In this embodiment of the application, the electronic device supports single-subject retrieval. This means that no matter how many subjects are included in the first image, the electronic device can retrieve a matching second image based on the subject that the user is interested in, thereby improving the accuracy of image retrieval.

[0076] Please refer to Figure 5 The diagram illustrates a flowchart of an image retrieval method according to an exemplary embodiment of this application. See also... Figure 5 The method includes:

[0077] Step 501: The electronic device receives an image retrieval instruction, which carries the first image.

[0078] In some embodiments, this step is the same as step 201, and will not be described again here.

[0079] Step 502: The electronic device performs subject segmentation on the first image to obtain multiple metadata of the first image, including multiple subjects included in the first image and attribute information of the multiple subjects.

[0080] In some embodiments, this step is the same as step 202, and will not be described again here.

[0081] Step 503: The electronic device displays multiple search options.

[0082] In some embodiments, this step is the same as step 403, and will not be described again here.

[0083] Step 504: When the multi-subject option among multiple search options is detected to be selected, the electronic device determines multiple second subjects from multiple metadata.

[0084] The electronic device displays an editing interface with multiple search options and subjects. Users can select search options and subjects based on this interface. In some embodiments, the user selects search options first and then subjects. This process can be as follows: the user can select search options by clicking; for example, the user clicks on a multi-subject option to select a multi-subject option (the first multi-subject option). The electronic device detects that the multi-subject option is selected. Then, the user clicks on multiple second subjects, and the electronic device identifies the selected second subjects. For example, if the user wants to search for two cups in the first image, the terminal can simply click on the two cups.

[0085] In other embodiments, the user first selects a subject and then selects a search option. This process can be as follows: the user can click on multiple second subjects and then click on the multi-subject option. At this time, the electronic device determines that the multi-subject option among the multiple search options is selected and identifies the selected multiple second subjects.

[0086] When the electronic device is configured as a server, the electronic device displays an editing interface and checks that the multi-subject option among the multiple search options in the editing interface is selected. It then determines multiple second subjects from multiple metadata and sends the subject identifiers of the multiple second subjects to the server. The server receives the subject identifiers of the multiple second subjects, determines the multiple second subjects based on the subject identifiers of the multiple second subjects, and then determines the search features of the first image based on the multiple second subjects through the following step 505.

[0087] Step 505: The electronic device combines the positional relationships of multiple second subjects into retrieval features of the first image.

[0088] The second subject includes not only the subject itself but also the subject's attribute information. In some embodiments, the terminal may also use only the positional relationship of multiple second subjects as the retrieval feature of the first image, in which case the feature category of the retrieval feature of the first image is subject relationship feature. In other embodiments, the terminal may also use only multiple second subjects as the retrieval feature of the first image, in which case the feature category of the retrieval feature of the first image is multi-subject feature.

[0089] Step 506: The electronic device retrieves a second image from the image library that matches the first image based on the retrieval features.

[0090] The image library includes multiple second images. For any second image, the electronic device extracts image features, including the subject, color, quantity, positional relationship, and other attribute information. The other attribute information is attribute information other than color and shape. Based on the retrieval features and the image features of the second image, the electronic device determines the similarity between the first image and the second image. For example, the similarity between the retrieval features and the image features of the second image is the same as the similarity between the first image and the second image. Based on this method, the similarity between the first image and multiple second images can be determined. Based on the similarity between the first image and multiple second images, the electronic device determines a second image from the image library whose similarity exceeds a second preset similarity. The determined second image is the second image that matches the first image. The number of second images can be one or multiple.

[0091] When an electronic device is configured as a server, after the server determines the second image, it sends the second image to the terminal. The terminal receives the second image and displays it, thereby presenting the image retrieval results to the user.

[0092] In this embodiment, the electronic device supports multi-subject retrieval, allowing users to select multiple subjects from multiple metadata of the first image to trigger a retrieval operation. As a result, the electronic device can retrieve a matching second image based on multiple subjects of interest to the user, thus improving the accuracy of image retrieval.

[0093] Please refer to Figure 6 The diagram illustrates a flowchart of an image retrieval method according to an exemplary embodiment of this application. See also... Figure 6 The method includes:

[0094] Step 601: The electronic device receives an image retrieval instruction, which carries the first image.

[0095] In some embodiments, this step is the same as step 201, and will not be described again here.

[0096] Step 602: The electronic device performs subject segmentation on the first image to obtain multiple metadata of the first image, including multiple subjects included in the first image and attribute information of the multiple subjects.

[0097] In some embodiments, this step is the same as step 202, and will not be repeated here. For example, please refer to Figure 7 The electronic device performs subject segmentation on the first image and obtains multiple metadata of the first image, including three water glasses and a table.

[0098] Step 603: The electronic device displays multiple search options.

[0099] In some embodiments, this step is the same as step 403, and will not be described again here.

[0100] Step 604: When the multi-subject option among multiple search options is detected to be selected, the electronic device determines multiple second subjects from multiple metadata.

[0101] In some embodiments, this step is the same as step 504, and will not be repeated here. Furthermore, after the electronic device identifies multiple second subjects, it determines the retrieval features of the first image based on the multiple second subjects through the following step 605. For example, please refer to... Figure 8 The electronic device identifies multiple secondary entities as three water cups from multiple metadata.

[0102] Step 605: When a drag operation is detected, the electronic device adjusts the positional relationship of multiple second subjects based on the drag operation.

[0103] Multiple second subjects have positional relationships, and users can drag the positional relationships of multiple second subjects to perform positional relationship retrieval. When a user wants to find an image with the same content but a different position, the embodiments of this application can provide more flexible options, and the scalability of subject relationship retrieval is enhanced by dragging the subject relationships. For example, please refer to... Figure 9 The first image contains three cups. The user wants to retrieve a second image that contains three cups, but the positional relationship of the three cups is not what the user wants. At this time, the user can drag the three cups to adjust their positional relationship, and then perform image retrieval based on the adjusted positional relationship of the three cups.

[0104] Step 606: The electronic device combines the adjusted multiple second subjects and the positional relationships of the adjusted multiple second subjects to form the retrieval features of the first image.

[0105] When the electronic device is configured as a server, the terminal displays an editing interface and detects that the multi-subject option among the multiple search options in the editing interface is selected. It then determines multiple second subjects from multiple metadata. When a drag operation is detected, the terminal adjusts the positional relationship of the multiple second subjects based on the drag operation and sends the subject identifiers of the multiple second subjects and the adjusted positional relationship of the multiple second subjects to the server. The server receives the subject identifiers of the multiple second subjects and the adjusted positional relationship of the multiple second subjects, and combines the adjusted multiple second subjects and the adjusted positional relationship of the multiple second subjects to form the retrieval features of the first image.

[0106] Step 607: The electronic device retrieves a second image from the image library that matches the first image based on the retrieval features.

[0107] The image library includes multiple second images. For any second image, the electronic device extracts image features, including the subject, color, quantity, positional relationship, and other attribute information. The other attribute information is attribute information other than color and shape. Based on the retrieval features and the image features of the second image, the electronic device determines the similarity between the first image and the second image. For example, the similarity between the retrieval features and the image features of the second image is the same as the similarity between the first image and the second image. Based on this method, the similarity between the first image and multiple second images can be determined. Based on the similarity between the first image and multiple second images, the electronic device determines a second image from the image library whose similarity exceeds a second preset similarity. The determined second image is the second image that matches the first image. The number of second images can be one or multiple.

[0108] When an electronic device is configured as a server, after the server determines the second image, it sends the second image to the terminal. The terminal receives the second image and displays it, thereby presenting the image retrieval results to the user.

[0109] In this embodiment of the application, when a user wants to find an image with the same content but a different location, this embodiment of the application can provide a more flexible choice, and enhance the scalability of subject relationship retrieval by dragging the subject relationship.

[0110] Please refer to Figure 10 The diagram illustrates a flowchart of an image retrieval method according to an exemplary embodiment of this application. See also... Figure 10 The method includes:

[0111] Step 1001: The electronic device receives an image retrieval instruction, which carries the first image.

[0112] In some embodiments, this step is the same as step 201, and will not be described again here.

[0113] Step 1002: The electronic device performs subject segmentation on the first image to obtain multiple metadata of the first image, including multiple subjects included in the first image and attribute information of the multiple subjects.

[0114] In some embodiments, this step is the same as step 202, and will not be described again here.

[0115] Step 1003: The electronic device displays multiple search options.

[0116] In some embodiments, this step is the same as step 403, and will not be described again here.

[0117] Step 1004: When an attribute option among multiple search options is detected to be selected, the electronic device determines a third subject from multiple metadata.

[0118] The electronic device displays an editing interface with multiple search options and multiple subjects. Users can select search options and subjects based on the editing interface. In some embodiments, users select search options first and then select subjects. This process can be as follows: users can select search options by clicking. For example, if a user clicks a single subject option to select an attribute option, the electronic device detects that the attribute option has been selected. Then, if the user clicks a third subject, the electronic device determines that the third subject has been selected.

[0119] In other embodiments, the user first selects a subject and then selects a search option. This process can be as follows: the user clicks on a third subject and then clicks on an attribute option. At this point, the electronic device determines that the attribute option among multiple search options is selected and identifies the selected third subject. For example, if the user is only interested in the shape of the third subject, the user can click on the shape option and then click on the third subject to trigger the electronic device to perform an image search based on the shape of the third subject.

[0120] After the electronic device identifies the third subject, it determines the retrieval features of the first image based on the attribute information of the third subject through the following step 1005.

[0121] Step 1005: When the attribute option is the shape option, the electronic device uses the shape contour features of the third subject as the retrieval features of the first image.

[0122] When the electronic device is configured as a server, the terminal displays an editing interface and checks that the attribute option among the multiple search options in the editing interface is selected. It then determines a third subject from multiple metadata, sends the subject identifier and shape identifier of the third subject to the server, receives the subject identifier and shape identifier of the third subject, and uses the shape contour features of the third subject as the search features of the first image based on the subject identifier and shape identifier of the third subject.

[0123] Step 1006: The electronic device retrieves a second image that matches the first image from the image library based on the retrieval features.

[0124] The image library includes multiple second images. For any second image, the electronic device extracts image features, including the shape and contour features of the main body included in the second image. Based on the retrieval features and the image features of the second image, the electronic device determines the similarity between the first image and the second image. For example, the similarity between the retrieval features and the image features of the second image is the same as the similarity between the first image and the second image. Based on this method, the similarity between the first image and multiple second images can be determined. Based on the similarity between the first image and multiple second images, the electronic device determines a second image from the image library whose similarity exceeds a third preset similarity. The determined second image is the second image that matches the first image. The number of second images can be one or multiple.

[0125] When an electronic device is configured as a server, after the server determines the second image, it sends the second image to the terminal. The terminal receives the second image and displays it, thereby presenting the image retrieval results to the user.

[0126] In this embodiment of the application, the electronic device supports shape retrieval, so that users can retrieve a second image containing the shape of an object they are interested in, thereby improving the flexibility and targeting of image retrieval.

[0127] Please refer to Figure 11 The diagram illustrates a flowchart of an image retrieval method according to an exemplary embodiment of this application. See also... Figure 11 The method includes:

[0128] Step 1101: The electronic device receives an image retrieval instruction, which carries the first image.

[0129] In some embodiments, this step is the same as step 201, and will not be described again here.

[0130] Step 1102: The electronic device performs subject segmentation on the first image to obtain multiple metadata of the first image. The multiple metadata includes multiple subjects included in the first image and attribute information of the multiple subjects.

[0131] In some embodiments, this step is the same as step 202, and will not be described again here.

[0132] Step 1103: The electronic device displays multiple search options.

[0133] In some embodiments, this step is the same as step 403, and will not be described again here.

[0134] Step 1104: When an attribute option among multiple search options is detected to be selected, the electronic device determines a third subject from multiple metadata.

[0135] In some embodiments, this step is the same as step 1004, and will not be repeated here. Furthermore, after the electronic device identifies the third subject, it determines the retrieval features of the first image based on the attribute information of the third subject through the following step 1105.

[0136] Step 1105: When the attribute option is color, the electronic device uses the color features of the third subject as the retrieval features of the first image.

[0137] When the electronic device is configured as a server, the terminal displays an editing interface. Upon detecting that an attribute option among the multiple search options in the editing interface is selected, it identifies a third subject from multiple metadata sources and sends the subject's identifier and color identifier to the server. The server receives the subject's identifier and color identifier and, based on these, uses the third subject's color feature as the search feature for the first image. For example, if the user is only interested in the color of the third subject, clicking the color option and then clicking on the third subject will trigger the electronic device to perform an image search based on the third subject's color.

[0138] Step 1106: The electronic device retrieves a second image that matches the first image from the image library based on the retrieval features.

[0139] The image library includes multiple second images. For any second image, the electronic device extracts image features, including the color features of the main subject included in the second image. Based on the retrieval features and the image features of the second image, the electronic device determines the similarity between the first image and the second image. For example, the similarity between the retrieval features and the image features of the second image is the same as the similarity between the first image and the second image. Based on this method, the similarity between the first image and multiple second images can be determined. Based on the similarity between the first image and multiple second images, the electronic device determines a second image from the image library whose similarity exceeds a fourth preset similarity. The determined second image is the second image that matches the first image. The number of second images can be one or multiple.

[0140] When an electronic device is configured as a server, after the server determines the second image, it sends the second image to the terminal. The terminal receives the second image and displays it, thereby presenting the image retrieval results to the user.

[0141] In this embodiment of the application, the electronic device supports color retrieval, so that users can retrieve a second image containing the color of an object they are interested in, thereby improving the flexibility and targeting of image retrieval.

[0142] Please refer to Figure 12 The diagram illustrates a flowchart of an image retrieval method according to an exemplary embodiment of this application. See also... Figure 12 The method includes:

[0143] Step 1201: The electronic device receives an image retrieval instruction, which carries the first image.

[0144] In some embodiments, this step is the same as step 201, and will not be described again here.

[0145] Step 1202: The electronic device performs subject segmentation on the first image to obtain multiple metadata of the first image. The multiple metadata includes multiple subjects included in the first image and attribute information of the multiple subjects.

[0146] In some embodiments, this step is the same as step 202, and will not be described again here.

[0147] Step 1203: The electronic device displays multiple search options.

[0148] In some embodiments, this step is the same as step 403, and will not be described again here.

[0149] Step 1204: When an attribute option among multiple search options is detected to be selected, the electronic device determines a third subject from multiple metadata.

[0150] In some embodiments, this step is the same as step 1104, and will not be repeated here. Furthermore, after the electronic device identifies the third subject, it determines the retrieval features of the first image based on the attribute information of the third subject through the following step 1205.

[0151] Step 1205: When the attribute option is set to "Other", the electronic device uses the other attribute features of the third subject as the retrieval features of the first image. The other attributes are attributes other than color and shape.

[0152] When the third subject is a user, the other attribute information of the third subject can be at least one of the user's actions (raising a hand or kicking a leg, etc.) and expressions (laughing or crying, etc.). When the third subject is an object, the other attribute information of the third subject can be the state of the object, etc.; for example, if the subject is a water cup, then the other attributes of the water cup can be whether the water cup has a lid, whether the water cup is tilted, etc.

[0153] When the electronic device is configured as a server, the terminal displays an editing interface and checks that other options among the multiple search options in the editing interface are selected. It then determines a third subject from multiple metadata, sends the subject identifier of the third subject and other identification information corresponding to the selection to the server, receives the subject identifier of the third subject and other identification information corresponding to the selection, and uses the other attribute features of the third subject as the search features of the first image based on the subject identifier of the third subject and other identification information corresponding to the selection.

[0154] Step 1206: The electronic device retrieves a second image that matches the first image from the image library based on the retrieval features.

[0155] The image library includes multiple second images. For any second image, the electronic device extracts image features, including other attribute features of the subject included in the second image. Based on the retrieval features and the image features of the second image, the electronic device determines the similarity between the first image and the second image. For example, the similarity between the retrieval features and the image features of the second image is the same as the similarity between the first image and the second image. Based on this method, the similarity between the first image and multiple second images can be determined. Based on the similarity between the first image and multiple second images, the electronic device determines a second image from the image library whose similarity exceeds a fifth preset similarity. The determined second image is the second image that matches the first image. The number of second images can be one or multiple.

[0156] When an electronic device is configured as a server, after the server determines the second image, it sends the second image to the terminal. The terminal receives the second image and displays it, thereby presenting the image retrieval results to the user.

[0157] In this embodiment of the application, the electronic device supports the retrieval of other attribute features, so that users can retrieve a second image containing other attribute features based on other attribute features of the object they are interested in, thereby improving the flexibility and targeting of image retrieval.

[0158] Please refer to Figure 13 This document illustrates a flowchart of an image retrieval method according to an exemplary embodiment of this application. The execution entity in this embodiment is an electronic device. See also... Figure 13The method includes:

[0159] Step 1301: The electronic device receives an image retrieval instruction, which carries the first image.

[0160] In some embodiments, this step is the same as step 201, and will not be described again here.

[0161] Step 1302: The electronic device performs subject segmentation on the first image to obtain multiple metadata of the first image, including multiple subjects included in the first image and attribute information of the multiple subjects.

[0162] In some embodiments, this step is the same as step 203, and will not be described again here.

[0163] Step 1303: The electronic device determines the retrieval features of the first image based on multiple metadata and target retrieval options. The retrieval features are used for similarity matching with other images, and the target retrieval options are used to determine the retrieval features from multiple metadata.

[0164] In some embodiments, this step can be implemented by steps 403-404 above; or, this step can be implemented by steps 503-505 above; or, this step can be implemented by steps 603-606 above; or, this step can be implemented by steps 1003-1005 above; or, this step can be implemented by steps 1103-1105 above; or, this step can be implemented by steps 1203-1205 above, which will not be described again here.

[0165] Step 1304: The electronic device obtains multiple metadata associated with the multiple second images based on the image identifiers of the multiple second images.

[0166] Before receiving an image search instruction, for any second image in the image library, the electronic device performs subject segmentation on the second image, obtains multiple metadata of the second image, and stores the image identifier and multiple metadata of the second image in association. Correspondingly, in this step, the electronic device directly retrieves the multiple metadata of the multiple second images stored in association based on the image identifiers of the multiple second images. For example, please refer to [link to relevant documentation]. Figure 3 The electronic device identifies the image library, and then uses the SAM model to perform subject segmentation on multiple second images in the image library, obtaining multiple metadata of the multiple second images. The multiple metadata of the multiple second images constitute the searched library.

[0167] In this embodiment, an index library is established for multiple second image segmentation metadata in the image library, preprocessing basic data information and saving post-processing time. The storage format of the image identifier and multiple metadata of the second image associated with the electronic device can be referred to in Table 1 below.

[0168] Table 1

[0169] In some embodiments, Table 1 above is an example of the metadata of the second image and does not limit the metadata of the second image. The metadata of the second image can be extended as needed. Furthermore, the process by which the electronic device performs subject segmentation on the second image to obtain multiple metadata of the second image is similar to the process by which the electronic device performs subject segmentation on the first image to obtain multiple metadata of the first image, and will not be described again here.

[0170] Step 1305: The electronic device determines multiple candidate images based on the metadata included in the search features and the metadata of multiple second images included in the image library, and the metadata of the multiple candidate images matches the metadata included in the search features.

[0171] For any second image, the electronic device determines the similarity between the first image and the second image based on the metadata included in the retrieval features and the metadata of the second image; if the similarity exceeds a sixth preset similarity, the second image is identified as a candidate image; if the similarity does not exceed the sixth preset similarity, the second image is not considered a candidate image. For example, please refer to [link / reference]. Figure 3 The electronic device performs coarse-grained retrieval based on search features and metadata from multiple second images.

[0172] In this embodiment, a coarse screening method is used to first identify multiple candidate images, and then a second image matching the first image is retrieved from these candidate images. This reduces the scope of image retrieval, thereby improving the efficiency of image retrieval. Furthermore, the coarse screening method also reduces the amount of image data required for subsequent preprocessing, thus improving the efficiency of image preprocessing.

[0173] Step 1306: The electronic device determines a second image that matches the first image from multiple candidate images based on retrieval features.

[0174] In some embodiments, after identifying multiple candidate images, the electronic device can directly determine a second image matching the first image from the multiple candidate images based on retrieval features, thereby improving the efficiency of image retrieval. In other embodiments, after identifying multiple candidate images, the electronic device can also preprocess the multiple candidate images before performing image retrieval; correspondingly, this step can be: the electronic device preprocesses the multiple candidate images based on the metadata included in the retrieval features; and based on the retrieval features, determines a second image matching the first image from the preprocessed multiple candidate images.

[0175] For example, please continue to refer to Figure 3 Preprocessing includes at least one of the following: size normalization, color space unification, image enhancement, and image rotation. For example, if the size of the subject included in the search features is inconsistent with the size of the subject included in the candidate image, the electronic device scales the subject included in the candidate image based on the size of the subject included in the search features to normalize the subject size. Similarly, if the color space of the subject included in the search features is inconsistent with the color space of the subject included in the candidate image, the electronic device converts the color space of the subject included in subsequent images based on the color space of the subject included in the search features to unify the color space. In this embodiment, by preprocessing multiple subsequent images, the influence of other external factors (such as size or color space) on the search results can be reduced, thereby improving the accuracy of image retrieval.

[0176] In some embodiments, the electronic device determines a second image matching the first image from a plurality of candidate images based on retrieval features through the following steps (1) to (4):

[0177] (1) Electronic devices determine the feature category of the search features based on the metadata included in the search features.

[0178] Feature categories can be single-subject features, multi-subject features, shape features, color features, and subject relationship features.

[0179] (2) Electronic devices extract image features from multiple candidate images based on feature categories.

[0180] When the feature category is multi-subject features, the electronic device extracts the subject, color, quantity, positional relationship, and other attribute information of the candidate image to obtain the image features of the candidate image. When the feature category is shape features, the electronic device extracts the shape and contour features of the candidate image to obtain the image features of the candidate image. When the feature category is color features, the electronic device extracts the color features of the candidate image to obtain the image features of the candidate image. When the feature category is subject relationship features, the electronic device extracts the positional relationship features of the subjects included in the candidate image to obtain the image features of the candidate image.

[0181] (3) The electronic device determines the similarity between the first image and multiple candidate images based on the retrieval features and the image features of multiple candidate images.

[0182] For any candidate image, the electronic device determines the similarity between the retrieved features and the image features of the candidate image, thus obtaining the similarity between the first image and the candidate image.

[0183] (4) The electronic device determines a second image that matches the first image from among the multiple candidate images based on the similarity between the first image and multiple candidate images.

[0184] The electronic device determines a second image from an image library whose similarity to a seventh preset similarity level is based on the similarity between a first image and multiple candidate images. The determined second image is the one that matches the first image. The number of second images can be one or more. For example, please refer to [link / reference]. Figure 3 The electronic device extracts features from multiple candidate images, then determines their similarity, and finally outputs the search results.

[0185] When an electronic device is configured as a server, after the server determines the second image, it sends the second image to the terminal. The terminal receives the second image and displays it, thereby presenting the image retrieval results to the user.

[0186] In this embodiment, a coarse screening method is used to first identify multiple candidate images, and then a second image matching the first image is retrieved from these candidate images. This reduces the scope of image retrieval, thereby improving the efficiency of image retrieval. Furthermore, the coarse screening method also reduces the amount of image data required for subsequent preprocessing, thus improving the efficiency of image preprocessing.

[0187] Please refer to Figure 14 The diagram illustrates a flowchart of an image retrieval method according to an exemplary embodiment of this application. See also... Figure 14 The method includes:

[0188] Step 1401: The electronic device receives an image comparison instruction, which carries a third image and a fourth image.

[0189] In some embodiments, the steps for an electronic device to receive an image comparison instruction are similar to those for an electronic device to receive an image retrieval instruction, and will not be described again here.

[0190] Step 1402: The electronic device performs subject segmentation on the third image and the fourth image respectively, and obtains multiple metadata of the third image and multiple metadata of the fourth image.

[0191] In some embodiments, the process by which an electronic device performs subject segmentation on a third image and a fourth image to obtain multiple metadata of the third image and multiple metadata of the fourth image is similar to the process by which an electronic device performs subject segmentation on a first image to obtain multiple metadata of the first image, and will not be described again here.

[0192] Step 1403: The electronic device determines the similarity between the third image and the fourth image based on multiple metadata of the third image and multiple metadata of the fourth image.

[0193] The electronic device determines the similarity between multiple metadata of the third image and multiple metadata of the fourth image, thus obtaining the similarity between the third image and the fourth image.

[0194] Step 1404: The electronic device determines the image comparison result between the third image and the fourth image based on the similarity between the third image and the fourth image, and the image comparison result is either similar or dissimilar.

[0195] If the similarity exceeds the eighth similarity level, the electronic device determines that the image comparison result is similar; if the similarity does not exceed the eighth similarity level, the electronic device determines that the image comparison result is dissimilar.

[0196] When an electronic device is configured as a server, after the server determines the second image, it sends the second image to the terminal. The terminal receives the second image and displays it, thereby presenting the image retrieval results to the user.

[0197] In this application embodiment, image segmentation and comparison based on the metadata obtained from the segmentation can be used to determine similar images. Since comparison based on metadata can improve the accuracy of the comparison, this application can improve the accuracy of similar image determination, thereby improving the user experience.

[0198] Please refer to Figure 15 The diagram illustrates a block diagram of an image retrieval apparatus according to an exemplary embodiment of this application, the apparatus comprising:

[0199] The receiving module 1501 is used to receive an image retrieval command, which carries a first image.

[0200] The segmentation module 1502 is used to perform subject segmentation on the first image to obtain multiple metadata of the first image, including multiple subjects included in the first image and attribute information of the multiple subjects;

[0201] The determination module 1503 is used to determine the retrieval features of the first image based on multiple metadata and target retrieval options. The retrieval features are used for similarity matching with other images, and the target retrieval options are used to determine the retrieval features from multiple metadata.

[0202] The retrieval module 1504 is used to retrieve a second image that matches the first image from the image library based on retrieval features.

[0203] In some embodiments, the determining module 1503 is configured to: if the target retrieval option is a single-subject option, determine a first subject from multiple metadata and use the first subject as a retrieval feature of the first image; if the target retrieval option is a multi-subject option, determine multiple second subjects from multiple metadata and determine the retrieval feature of the first image based on the multiple second subjects; if the target retrieval option is an attribute option, determine a third subject from multiple metadata and determine the retrieval feature of the first image based on the attribute information of the third subject.

[0204] In some embodiments, the determining module 1503 is configured to combine the positional relationships of the plurality of second subjects into retrieval features of the first image; or,

[0205] The determination module 1503 is used to adjust the positional relationship of multiple second subjects based on the drag operation when a drag operation is detected; and to form the retrieval features of the first image by combining the adjusted multiple second subjects and the adjusted positional relationship of the multiple second subjects.

[0206] In some embodiments, the determining module 1503 is configured to, when the attribute option is a shape option, use the shape contour features of the third subject as the retrieval features of the first image; when the attribute option is a color option, use the color features of the third subject as the retrieval features of the first image; and when the attribute option is another option, use other attribute features of the third subject as the retrieval features of the first image, wherein the other attributes are attributes other than color and shape.

[0207] In some embodiments, the retrieval module 1504 is configured to determine a plurality of candidate images based on the metadata included in the retrieval features and the metadata of a plurality of second images included in the image library, wherein the metadata of the plurality of candidate images matches the metadata included in the retrieval features; and to determine a second image that matches the first image from the plurality of candidate images based on the retrieval features.

[0208] In some embodiments, the apparatus further includes:

[0209] The segmentation module 1502 is also used to perform subject segmentation on any second image in the image library before receiving an image search instruction, obtain multiple metadata of the second image, and associate and store the image identifier of the second image and the multiple metadata of the second image.

[0210] The acquisition module is used to acquire multiple metadata associated with multiple second images based on the image identifiers of multiple second images.

[0211] In some embodiments, the retrieval module 1504 is configured to determine the feature category of the retrieval feature based on the metadata included in the retrieval feature; extract image features of multiple candidate images based on the feature category; determine the similarity between a first image and multiple candidate images based on the retrieval feature and the image features of multiple candidate images; and determine a second image that matches the first image from the multiple candidate images based on the similarity between the first image and multiple candidate images.

[0212] In some embodiments, the retrieval module 1504 is configured to preprocess a plurality of candidate images based on metadata included in the retrieval features; and to determine a second image that matches the first image from the preprocessed plurality of candidate images based on the retrieval features.

[0213] In some embodiments, the retrieval module 1504 is used to extract the subject, color and other attribute information of any candidate image among a plurality of candidate images, when the feature category is a single subject feature, to obtain the image features of the candidate image, wherein the other attributes are attributes other than color and shape.

[0214] The retrieval module 1504 is used to extract the subject, color, quantity, positional relationship and other attribute information of the candidate image when the feature category is multi-subject features, so as to obtain the image features of the candidate image. Other attributes are attributes other than color and shape.

[0215] The retrieval module 1504 is used to extract the shape contour features of the candidate image when the feature category is shape features, so as to obtain the image features of the candidate image;

[0216] The retrieval module 1504 is used to extract the color features of the candidate image when the feature category is color features, so as to obtain the image features of the candidate image;

[0217] The retrieval module 1504 is used to extract the positional relationship features of the subjects included in the candidate image when the feature category is subject relationship features, so as to obtain the image features of the candidate image.

[0218] In some embodiments, the apparatus further includes:

[0219] The receiving module 1501 is also used to receive an image comparison instruction, which carries a third image and a fourth image;

[0220] The segmentation module 1502 is also used to perform subject segmentation on the third image and the fourth image respectively, and obtain multiple metadata of the third image and multiple metadata of the fourth image;

[0221] The determination module 1503 is also used to determine the similarity between the third image and the fourth image based on multiple metadata of the third image and multiple metadata of the fourth image;

[0222] The comparison module is used to determine the image comparison result between the third image and the fourth image based on the similarity between the two images. The image comparison result is either similar or dissimilar.

[0223] In some embodiments, the determining module 1503 is used to display multiple search options; when it is detected that a target search option among the multiple search options is selected, the search features of the first image are determined based on multiple metadata and the target search option.

[0224] In this embodiment, on the one hand, by performing subject segmentation on the first image, multiple metadata of the first image are obtained, including multiple subjects included in the first image and attribute information of the subjects. Therefore, this application can obtain more granular metadata. On the other hand, by using target retrieval options, retrieval features can be determined from the multiple metadata, thus enabling image retrieval to be more targeted. Based on the above two points, it can be seen that this application can perform image retrieval in a more granular and targeted manner, thereby improving the accuracy of image retrieval.

[0225] It should be noted that the image retrieval device provided in the above embodiments is only illustrated by the division of the above functional modules when performing image retrieval. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the terminal can be divided into different functional modules to complete all or part of the functions described above. In addition, the image retrieval device and the image retrieval method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0226] If the electronic device is configured as a terminal, please refer to Figure 10 The diagram illustrates a block diagram of a terminal 1600 according to an exemplary embodiment of this application. The terminal 1600 in this application may include one or more components such as a processor 1610, a memory 1620, and a touchscreen 1630.

[0227] Processor 1610 may include one or more processing cores. Processor 1610 connects to various parts within terminal 1600 using various interfaces and lines, and performs various functions and processes data of terminal 1600 by running or executing instructions, programs, code sets, or instruction sets stored in memory 1620, and by calling data stored in memory 1620. Optionally, processor 1610 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 1610 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), and modem. Specifically, the CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on touchscreen 1630; the NPU is used to implement artificial intelligence (AI) functions; and the modem is used for wireless communication. It is understandable that the aforementioned modem may not be integrated into the processor 1610, but may be implemented as a separate chip.

[0228] The memory 1620 may include random access memory (RAM) or read-only memory. Optionally, the memory 1620 may include a non-transitory computer-readable storage medium. The memory 1620 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1620 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc.; the data storage area may store data created based on the use of the terminal 1600 (such as audio data, phone book, etc.).

[0229] Touchscreen 1630 is a display component used to display a user interface. Optionally, touchscreen 1630 is a touch-enabled display screen, through which users can use their fingers, styluses, or any suitable object to perform touch operations on touchscreen 1630.

[0230] The touchscreen 1630 is typically located on the front panel of the terminal 1600. The touchscreen 1630 can be designed as a full-screen, curved screen, irregularly shaped screen, dual-sided screen, or foldable screen. The touchscreen 1630 can also be designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen, etc., but this embodiment does not limit it in this way.

[0231] In addition, those skilled in the art will understand that the structure of the terminal 1600 shown in the above figures does not constitute a limitation on the terminal 1600. The terminal 1600 may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the terminal 1600 may also include audio acquisition devices, speakers, radio frequency circuits, input units, sensors, audio circuits, a Wireless Fidelity (Wi-Fi) module, a power supply, a Bluetooth module, and other components, which will not be described in detail here.

[0232] If the electronic device is configured as a server, please refer to Figure 17 This diagram illustrates a block diagram of a server according to an exemplary embodiment of this application. Servers can vary significantly due to differences in configuration or performance, and may include a central processing unit (CPU) 1701 and a memory 1702. The memory 1702 stores at least one line of program code, which is loaded and executed by the processor 1701 to implement the methods provided in the various method embodiments described above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input / output. The server may also include other components for implementing device functions, which will not be elaborated upon here.

[0233] This application also provides a computer-readable medium storing at least one piece of program code, which is loaded and executed by the processor to implement the image retrieval method shown in the above embodiments.

[0234] This application also provides a computer program product that stores at least one piece of program code, which is loaded and executed by the processor to implement the image retrieval method shown in the above embodiments.

[0235] In some embodiments, the computer program product involved in the present application can be deployed on an electronic device for execution, or executed on multiple electronic devices located in one location, or executed on multiple electronic devices distributed in multiple locations and interconnected through a communication network. Multiple electronic devices distributed in multiple locations and interconnected through a communication network can form a blockchain system.

[0236] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0237] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image retrieval method, characterized in that, The method includes: Receive an image retrieval instruction, wherein the image retrieval instruction carries a first image; The first image is segmented to obtain multiple metadata of the first image, including multiple subjects included in the first image and attribute information of the multiple subjects; Based on the plurality of metadata and target retrieval options, the retrieval features of the first image are determined. The retrieval features are used to perform similarity matching with other images. The target retrieval options are used to determine the retrieval features from the plurality of metadata. The target retrieval options include any one of single subject options, multi-subject options, and attribute options. Based on the metadata included in the search features and the metadata of multiple second images included in the image library, multiple candidate images are determined, and the metadata of the multiple candidate images is matched with the metadata included in the search features; based on the search features, a second image matching the first image is determined from the multiple candidate images. The step of determining a second image matching the first image from the plurality of candidate images based on the retrieval features includes: Based on the metadata included in the retrieval features, the feature category of the retrieval features is determined; based on the feature category, image features of the plurality of candidate images are extracted; based on the retrieval features and the image features of the plurality of candidate images, the similarity between the first image and the plurality of candidate images is determined; based on the similarity between the first image and the plurality of candidate images, a second image matching the first image is determined from the plurality of candidate images.

2. The method according to claim 1, characterized in that, The determination of the retrieval features of the first image based on the multiple metadata and target retrieval options includes any of the following implementation methods: If the target retrieval option is a single subject option, then a first subject is determined from the plurality of metadata, and the first subject is used as the retrieval feature of the first image; If the target retrieval option is a multi-subject option, then multiple second subjects are determined from the multiple metadata, and the retrieval features of the first image are determined based on the multiple second subjects; If the target retrieval option is an attribute option, then a third subject is determined from the plurality of metadata, and the retrieval features of the first image are determined based on the attribute information of the third subject.

3. The method according to claim 2, characterized in that, The step of determining the retrieval features of the first image based on the plurality of second subjects includes: The retrieval features of the first image are formed by combining the positional relationships of the plurality of second subjects; or, When a drag operation is detected, the positional relationship of the plurality of second subjects is adjusted based on the drag operation; the adjusted plurality of second subjects and the adjusted positional relationship of the plurality of second subjects are combined to form the retrieval features of the first image.

4. The method according to claim 2, characterized in that, The determination of the retrieval features of the first image based on the attribute information of the third subject includes any of the following implementation methods: When the attribute option is the shape option, the shape contour features of the third subject are used as the retrieval features of the first image; When the attribute option is the color option, the color feature of the third subject is used as the retrieval feature of the first image; When the attribute option is set to "Other", the other attribute features of the third subject are used as the retrieval features of the first image. The other attributes are attributes other than color and shape.

5. The method according to claim 1, characterized in that, The method further includes: Before receiving the image retrieval instruction, for any second image in the image library, the second image is segmented to obtain multiple metadata of the second image, and the image identifier of the second image and the multiple metadata of the second image are stored together. Before determining the plurality of candidate images, multiple metadata associated with the plurality of second images are obtained based on the image identifiers of the plurality of second images.

6. The method according to claim 1, characterized in that, The step of determining a second image matching the first image from the plurality of candidate images based on the retrieval features includes: Based on the metadata included in the retrieval features, the multiple candidate images are preprocessed; Based on the retrieval features, a second image matching the first image is determined from the preprocessed plurality of candidate images.

7. The method according to claim 1, characterized in that, The step of extracting image features from the multiple candidate images based on the feature categories includes any of the following implementation methods: For any candidate image among the plurality of candidate images, when the feature category is a single subject feature, the subject, color and other attribute information of the candidate image are extracted to obtain the image features of the candidate image, wherein the other attributes are attributes other than color and shape; When the feature category is a multi-subject feature, the subject, color, quantity, positional relationship and other attribute information of the candidate image are extracted to obtain the image features of the candidate image. The other attributes are attributes other than color and shape. When the feature category is shape feature, the shape contour feature of the candidate image is extracted to obtain the image feature of the candidate image; When the feature category is color feature, extract the color feature of the candidate image to obtain the image feature of the candidate image; If the feature category is subject relationship feature, the positional relationship features of the subjects included in the candidate image are extracted to obtain the image features of the candidate image.

8. The method according to claim 1, characterized in that, The method further includes: Receive an image comparison instruction, the image comparison instruction carrying a third image and a fourth image; Subject segmentation is performed on the third image and the fourth image respectively to obtain multiple metadata of the third image and multiple metadata of the fourth image; Based on multiple metadata of the third image and multiple metadata of the fourth image, the similarity between the third image and the fourth image is determined; Based on the similarity between the third image and the fourth image, an image comparison result between the third image and the fourth image is determined, and the image comparison result is either similar or dissimilar.

9. The method according to claim 1, characterized in that, The step of determining the retrieval features of the first image based on the multiple metadata and target retrieval options includes: Displays multiple search options; When the target search option among the plurality of search options is detected to be selected, the search features of the first image are determined based on the plurality of metadata and the target search option.

10. An image retrieval device, characterized in that, The device includes: A receiving module is configured to receive an image retrieval instruction, wherein the image retrieval instruction carries a first image; The segmentation module is used to perform subject segmentation on the first image to obtain multiple metadata of the first image, wherein the multiple metadata includes multiple subjects included in the first image and attribute information of the multiple subjects; The determining module is used to determine the retrieval features of the first image based on the plurality of metadata and target retrieval options. The retrieval features are used for similarity matching with other images. The target retrieval options are used to determine the retrieval features from the plurality of metadata. The target retrieval options include any one of single subject options, multi-subject options, and attribute options. The retrieval module is configured to determine multiple candidate images based on the metadata included in the retrieval features and the metadata of multiple second images included in the image library, wherein the metadata of the multiple candidate images matches the metadata included in the retrieval features; and to determine a second image that matches the first image from the multiple candidate images based on the retrieval features. The retrieval module is further configured to: determine the feature category of the retrieval feature based on the metadata included in the retrieval feature; extract image features of the plurality of candidate images based on the feature category; determine the similarity between the first image and the plurality of candidate images based on the retrieval feature and the image features of the plurality of candidate images; and determine a second image matching the first image from the plurality of candidate images based on the similarity between the first image and the plurality of candidate images.

11. The apparatus according to claim 10, characterized in that, The determining module is configured to: if the target retrieval option is a single-subject option, determine a first subject from the plurality of metadata and use the first subject as the retrieval feature of the first image; if the target retrieval option is a multi-subject option, determine a plurality of second subjects from the plurality of metadata and determine the retrieval feature of the first image based on the plurality of second subjects; if the target retrieval option is an attribute option, determine a third subject from the plurality of metadata and determine the retrieval feature of the first image based on the attribute information of the third subject.

12. The apparatus according to claim 11, characterized in that, The determining module is used to form the retrieval features of the first image by combining the positional relationships of the plurality of second subjects; or, The determining module is used to adjust the positional relationship of the plurality of second entities based on the dragging operation when a dragging operation is detected. The adjusted plurality of second subjects and the adjusted positional relationships of the plurality of second subjects constitute the retrieval features of the first image.

13. The apparatus according to claim 11, characterized in that, The determining module is configured to, when the attribute option is a shape option, use the shape contour features of the third subject as the retrieval features of the first image; when the attribute option is a color option, use the color features of the third subject as the retrieval features of the first image; and when the attribute option is another option, use other attribute features of the third subject as the retrieval features of the first image, wherein the other attributes are attributes other than color and shape.

14. The apparatus according to claim 10, characterized in that, The device further includes: The segmentation module is further configured to, before receiving the image retrieval instruction, perform subject segmentation on any second image in the image library to obtain multiple metadata of the second image, and associate and store the image identifier of the second image with the multiple metadata of the second image; The acquisition module is used to acquire multiple metadata associated with the multiple second images based on the image identifiers of the multiple second images before determining the multiple candidate images.

15. The apparatus according to claim 10, characterized in that, The retrieval module is used to preprocess the plurality of candidate images based on the metadata included in the retrieval features; and to determine a second image that matches the first image from the preprocessed plurality of candidate images based on the retrieval features.

16. The apparatus according to claim 10, characterized in that, The retrieval module is used to extract the subject, color, and other attribute information of any candidate image among the plurality of candidate images, when the feature category is a single subject feature, to obtain the image features of the candidate image, wherein the other attributes are attributes other than color and shape; The retrieval module is used to extract the subject, color, quantity, positional relationship and other attribute information of the candidate image when the feature category is multi-subject feature, so as to obtain the image features of the candidate image, wherein the other attributes are attributes other than color and shape; The retrieval module is used to extract the shape contour features of the candidate image when the feature category is shape feature, so as to obtain the image features of the candidate image; The retrieval module is used to extract the color features of the candidate image when the feature category is color feature, so as to obtain the image features of the candidate image; The retrieval module is used to extract the positional relationship features of the subjects included in the candidate image when the feature category is subject relationship features, so as to obtain the image features of the candidate image.

17. The apparatus according to claim 10, characterized in that, The device further includes: The receiving module is also used to receive an image comparison instruction, which carries a third image and a fourth image; The segmentation module is further configured to perform subject segmentation on the third image and the fourth image respectively, to obtain multiple metadata of the third image and multiple metadata of the fourth image; The determining module is further configured to determine the similarity between the third image and the fourth image based on multiple metadata of the third image and multiple metadata of the fourth image; The comparison module is used to determine the image comparison result between the third image and the fourth image based on the similarity between the third image and the fourth image, wherein the image comparison result is similar or dissimilar.

18. The apparatus according to claim 10, characterized in that, The determining module is used to display multiple search options; when it is detected that a target search option among the multiple search options is selected, it determines the search features of the first image based on the multiple metadata and the target search option.

19. An electronic device, characterized in that, The electronic device includes one or more processors and one or more memories, wherein at least one piece of program code is stored in the one or more memories, and the at least one piece of program code is loaded and executed by the one or more processors to implement the image retrieval method as described in any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that, The storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the image retrieval method as described in any one of claims 1 to 9.

21. A computer program product, characterized in that, The computer program product stores at least one piece of program code, which is executed by a processor to implement the image retrieval method as described in any one of claims 1 to 9.

Images