Method for searching according to picture style

Abstract

The invention relates to a method for searching according to a picture style. The method comprises the following steps: S1, acquiring a picture uploaded by a user; s2, reasoning the uploaded picture by using an image coding model of a multi-modal coding model to obtain a mixed embedded vector; S3, reasoning the uploaded picture by using a cue word reverse deduction model to obtain a text describing the content of the picture; s4, inputting the text to a text coding model of a multi-modal coding model for reasoning to obtain an embedded vector of the object; and S5, subtracting the embedding vector of the object from the mixed embedding vector to obtain an embedding vector of the style. According to the method, the object information and the style information can be decoupled from the picture uploaded by the user so as to realize on-demand search.

Description

Technical Field

[0001] The present invention relates to the technical field of image search, and in particular to a method for searching according to image style. Background Art

[0002] Existing image search technologies are usually based on a multimodal coding model (such as CLIP), which encodes the entire image into a feature vector and compares it with other feature vectors in a vector database to find several images with the best similarity. The multimodal coding model used by existing technologies will mix the object information (such as main content and shape) and style information (such as color distribution and texture style) of the image, resulting in search results being affected by both the object and style. For example, when a user wants to search only for images that are "object-consistent" with the uploaded image, the existing algorithm may search for images that are more relevant in style rather than images that are more relevant in object; conversely, if the user wants to search for images that are "consistent in style", the existing algorithm will return images that are relevant in object but not in style. This hybrid coding method lacks flexibility and cannot meet the user's fine-grained search needs.

[0003] Therefore, it is necessary to provide a method for searching by image style, which can decouple object information and style information from user-uploaded images to achieve on-demand search. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for searching by picture style, which decouples object information and style information from pictures uploaded by users to achieve on-demand search.

[0005] In order to solve the problems existing in the prior art, the present invention provides a method for searching by image style, comprising the following steps:

[0006] S1: Get the pictures uploaded by the user;

[0007] S2: Use the image coding model of the multimodal coding model to infer the uploaded image and obtain a mixed embedding vector:

[0008] S3: Use the prompt word inference model to infer the uploaded image and obtain the text describing the content of the image;

[0009] S4: Inputting the text into a text encoding model of a multimodal encoding model for inference to obtain an embedding vector of the object;

[0010] S5: Subtract the object embedding vector from the mixed embedding vector to obtain the style embedding vector.

[0011] Optionally, in the method of searching according to picture style,

[0012] The multimodal encoding model is Contrastive Language-Image Pre-training.

[0013] Optionally, in the method of searching according to picture style,

[0014] The multimodal coding model includes an image coding model and a text coding model.

[0015] Compared with the prior art, the present invention has the following advantages:

[0016] (1) The present invention provides search flexibility: object information and style information are decoupled from user-uploaded images to achieve on-demand search; users can select search dimensions (object / style / comprehensive) according to their needs, thus overcoming the "one-size-fits-all" limitation of traditional algorithms.

[0017] (2) The present invention can be applied to image search in the AIGC painting community, providing a technical basis for scenarios such as style transfer and material recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of searching by image style provided by an embodiment of the present invention;

[0019] Figure 2 This is a display diagram of the image search page provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following is a more detailed description of the specific embodiments of the present invention with reference to schematic diagrams. The advantages and features of the present invention will become more apparent from the following description. It should be noted that the drawings are in a very simplified form and are not to exact scale, and are only used for the purpose of conveniently and clearly illustrating the embodiments of the present invention.

[0021] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0022] Existing image search technologies are usually based on a multimodal coding model (such as CLIP), which encodes the entire image into a feature vector and compares it with other feature vectors in a vector database to find several images with the best similarity. The multimodal coding model used by existing technologies will mix the object information (such as main content and shape) and style information (such as color distribution and texture style) of the image, resulting in search results being affected by both the object and style. For example, when a user wants to search only for images that are "object-consistent" with the uploaded image, the existing algorithm may search for images that are more relevant in style rather than images that are more relevant in object; conversely, if the user wants to search for images that are "consistent in style", the existing algorithm will return images that are relevant in object but not in style. This hybrid coding method lacks flexibility and cannot meet the user's fine-grained search needs.

[0023] In order to solve the problems existing in the prior art, the present invention proposes a method for searching according to image style, such as Figure 1 As shown, the room includes the following steps:

[0024] S1: Get the pictures uploaded by the user;

[0025] S2: Use the image coding model of the multimodal coding model to infer the uploaded image and obtain a mixed embedding vector:

[0026] The multimodal encoding model is Contrastive Language-Image Pre-training, which includes image encoding model and text encoding model.

[0027] S3: Use the prompt word inference model to infer the uploaded image and obtain the text describing the content of the image;

[0028] S4: Inputting the text into a text encoding model of a multimodal encoding model for inference to obtain an embedding vector of the object;

[0029] S5: Because the image feature vector couples object information and style information, and the present invention can provide object information through the text feature vector, the style information can be separated through vector operations, and the style embedding vector is obtained by subtracting the object embedding vector from the mixed embedding vector.

[0030] like Figure 2 As shown, Figure 2 This is a display diagram of the image search page provided by an embodiment of the present invention.

[0031] In summary, the present invention has the following advantages compared with the prior art:

[0032] (1) The present invention provides search flexibility: object information and style information are decoupled from user-uploaded images to achieve on-demand search; users can select search dimensions (object / style / comprehensive) according to their needs, thus overcoming the "one-size-fits-all" limitation of traditional algorithms.

[0033] (2) The present invention can be applied to image search in the AIGC painting community, providing a technical basis for scenarios such as style transfer and material recommendation.

[0034] The above description is merely a preferred embodiment of the present invention and does not limit the present invention in any way. Any person skilled in the art who, without departing from the scope of the present invention, makes any equivalent substitution, modification, or other changes to the technical solution and technical content disclosed in the present invention shall be deemed to be within the scope of the present invention and still fall within the scope of protection of the present invention.

Claims

1. A method for searching by picture style, characterized in that: The following steps are involved: S1: Get the pictures uploaded by the user; S2: Use the image coding model of the multimodal coding model to infer the uploaded image and obtain a mixed embedding vector: S3: Use the prompt word inference model to infer the uploaded image and obtain the text describing the content of the image; S4: Inputting the text into a text encoding model of a multimodal encoding model for inference to obtain an embedding vector of the object; S5: Subtract the object embedding vector from the mixed embedding vector to obtain the style embedding vector.

2. The method for searching by picture style according to claim 1, wherein: The multimodal encoding model is Contrastive Language-Image Pre-training.

3. The method for searching by picture style according to claim 1, wherein: The multimodal coding model includes an image coding model and a text coding model.

Images