Data processing method and device, computer device, and storage medium

Abstract

The application discloses a data processing method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining the expected image style and the expected content description of the expected image of a user; generating an image based on the expected content description and the expected image style to obtain an initial generated image, and extracting the semantic of the expected content description to obtain key image information; identifying the content of the initial generated image to obtain content identification data, and identifying the style of the initial generated image to obtain image identification style; when the image identification style is consistent with the expected image style, and the content identification data matches the key image information, outputting the initial generated image as a target search image corresponding to the expected image. In the application, the user can search for more satisfactory image search data, improve the image search effect, and improve the flexibility of image data acquisition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a data processing method and device, computer equipment and a storage medium. BACKGROUND

[0002] In recent years, with the rapid development of the Internet, how to quickly find the information needed by the user has become a key problem in the utilization and management of big data, thereby deriving an artificial intelligence system based on Internet big data. In the artificial intelligence system, people usually adopt a retrieval method to find useful information, such as text retrieval, image retrieval, etc.

[0003] Taking image retrieval as an example, the commonly used retrieval methods at present include text keyword-based retrieval and input image-based retrieval. The text keyword-based retrieval is that the user inputs a query text, and the system matches the user query text with the text label of the image data, thereby obtaining the image retrieval result. The input image-based retrieval is that the user inputs a standard image, and the system performs similarity calculation on the standard image and the image data, thereby obtaining the image retrieval result. However, these two retrieval methods can only provide the image data already existing in the database for the user, and a large amount of manual annotation or calculation needs to be performed on the image data set, which limits the flexibility of image data acquisition, and it is also difficult to obtain a relatively ideal image retrieval result, and the image retrieval effect is poor. SUMMARY

[0004] The present application provides a data processing method and device, computer equipment and a storage medium to solve the problem that only existing image data can be provided for the user in the existing image retrieval process, the flexibility of image data acquisition is low, and the image retrieval effect is poor.

[0005] A data processing method is provided, comprising:

[0006] Obtaining image description information of a desired image from a user, the image description information including desired image style and desired content description;

[0007] Generating an initial generated image based on the desired content description and the desired image style, and performing semantic extraction on the desired content description to obtain key image information corresponding to the desired content description;

[0008] Performing image content recognition on the initial generated image to obtain content recognition data, and performing style recognition on the initial generated image to obtain image recognition style;

[0009] Determining whether the image recognition style is consistent with the desired image style, and determining whether the content recognition data matches the key image information;

[0010] If the image recognition style is consistent with the expected image style, and the content recognition data matches the key image information, the initial generated image is output as a target search image corresponding to the expected image.

[0011] Optionally, the key image information includes a plurality of expected elements of the expected image, and the determining whether the content recognition data matches the key image information includes:

[0012] performing element recognition on the content recognition data to obtain a plurality of constituent elements of the initial generated image;

[0013] matching the plurality of expected elements with the plurality of constituent elements one by one to obtain a matching result;

[0014] If the element matching result is complete matching, it is determined that the content recognition data matches the key image information;

[0015] If the element matching result is not complete matching, it is determined that the content recognition data does not match the key image information.

[0016] Optionally, the key image information further includes a position and a color of each expected element, and the matching the plurality of expected elements with the plurality of constituent elements one by one to obtain a matching result includes:

[0017] determining whether the plurality of constituent elements and the plurality of expected elements can be matched one by one;

[0018] If the plurality of constituent elements and the plurality of expected elements can be matched one by one, determining a position and a color of each constituent element based on the content recognition data;

[0019] determining whether the position and the color of each constituent element match a position and a color of a corresponding expected element;

[0020] If the position and the color of each constituent element match the position and the color of the corresponding expected element, it is determined that the matching result is complete matching;

[0021] If the plurality of constituent elements and the plurality of expected elements cannot be matched one by one, or the position or the color of any constituent element does not match the position or the color of the corresponding expected element, it is determined that the matching result is not complete matching.

[0022] Optionally, the determining whether the position and the color of each constituent element match the position and the color of the corresponding expected element includes:

[0023] calculating a distance between the position of each constituent element and the position of the corresponding expected element to obtain a position distance of each constituent element;

[0024] calculating a similarity between the color of each constituent element and the color of the corresponding expected element to obtain a color difference of each constituent element;

[0025] determining whether the position distance of each component element is less than a preset distance, and determining whether the color difference of each component element is less than a preset color difference value;

[0026] If the position distance of each component element is less than the preset distance, and the color difference of each component element is less than the preset color difference value, it is determined that the position and color of each component element match the position and color of the corresponding expected element.

[0027] Optionally, the key image information further includes an expected background description, and after one-to-one matching of the plurality of expected elements and the plurality of component elements is performed to obtain a matching result, the method further includes:

[0028] If the matching result is a complete match, the content recognition data is subjected to background recognition to obtain an image background description of the initial generated image;

[0029] determining a background similarity between the expected background description and the image background description;

[0030] If the background similarity is greater than or equal to a preset background similarity, it is determined that the content recognition data matches the key image information;

[0031] If the background similarity is less than the preset background similarity, it is determined that the content recognition data does not match the key image information.

[0032] Optionally, the expected background description includes an expected background color and an expected background pattern, the image background description includes a recognized background color and a recognized background pattern, and the determination of the background similarity between the expected background description and the image background description includes:

[0033] calculating a background color similarity between the expected background color and the recognized background color, and calculating a pattern similarity between the expected background pattern and the recognized background pattern;

[0034] calculating the background similarity based on the background color similarity and the pattern similarity.

[0035] Optionally, the image generation based on the expected content description and the expected image style to obtain the initial generated image includes:

[0036] obtaining a target image generation model obtained by deep learning of a generative adversarial network based on a model sample data set, each historical image sample in the model sample data set corresponding to a sample content description vector and a sample style vector, and the target image generation model being obtained by deep learning training of the generative adversarial network based on the sample content description vectors and the sample style vectors of a plurality of historical image samples;

[0037] performing semantic feature extraction on the expected content description to obtain an image description vector, and performing semantic feature extraction on the expected image style to obtain an expected image style vector.

[0038] The desired image style vector and the image description vector are input into the target image generation model for image generation, and an image output by the target image generation model is obtained as an initial generated image.

[0039] A data processing apparatus is provided, comprising:

[0040] An acquisition module is configured to acquire image description information of a desired image from a user, the image description information including a desired image style and a desired content description;

[0041] A generation module is configured to perform image generation based on the desired content description and the desired image style to obtain an initial generated image, and perform semantic extraction on the desired content description to obtain key image information corresponding to the desired content description;

[0042] An identification module is configured to perform image content identification on the initial generated image to obtain content identification data, and perform style identification on the initial generated image to obtain an image identification style;

[0043] A determination module is configured to determine whether the image identification style is consistent with the desired image style, and whether the content identification data matches the key image information;

[0044] An output module is configured to output the initial generated image as a target search image corresponding to the desired image if the image identification style is consistent with the desired image style, and the content identification data matches the key image information.

[0045] A computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the data processing method when executing the computer program.

[0046] A computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the data processing method when executed by a processor.

[0047] In one scheme of the data processing method, the data processing device, the computer device, and the storage medium, the image description information of the expected image is obtained, the image description information includes the expected image style and the expected content description, the image is generated based on the expected content description and the expected image style to obtain an initial generated image, the semantic extraction is performed on the expected content description to obtain the key image information corresponding to the expected content description, the image content recognition is performed on the initial generated image to obtain content recognition data, the style recognition is performed on the initial generated image to obtain an image recognition style, whether the image recognition style is consistent with the expected image style is determined, and whether the content recognition data matches the key image information is determined, if the image recognition style is consistent with the expected image style and the content recognition data matches the key image information, the initial generated image is output as the target retrieval image corresponding to the expected image. In the application, when the user performs the expected image retrieval, the initial generated image is generated based on the expected image style and the expected content description input by the user, and when it is determined that the initial generated image meets the image description information of the expected image of the user, the initial generated image is output as the image retrieval result. The user can retrieve more satisfactory image retrieval data, the image retrieval effect is improved, and the flexibility of image data acquisition is improved. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the application. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Figure 1 is an application environment schematic diagram of the data processing method in an embodiment of the application;

[0050] Figure 2 is a flow schematic diagram of the data processing method in an embodiment of the application;

[0051] Figure 3 is Figure 2 is an implementation flow schematic diagram of step S20 in the method;

[0052] Figure 4 is Figure 2 is an implementation flow schematic diagram of step S40 in the method;

[0053] Figure 5 is Figure 2 is another implementation flow schematic diagram of step S40 in the method;

[0054] Figure 6 is a structure schematic diagram of the data processing method in an embodiment of the application;

[0055] Figure 7 Figure 1 is a structural schematic diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0057] The data processing method provided by the embodiments of the present application can be applied in an application scenario as shown in the figure. Figure 1 The server and the terminal device communicate through the network. The server can be a background server of a preset editing platform, which is a platform providing design functions of visual digital products such as H5 pages or banners. When a user needs to perform image retrieval for visual digital product design, the user can input image description information of an expected image through the terminal device, the image description information including expected image style and expected content description of the expected image, so that the server performs image generation and retrieval tasks based on the image description information, thereby generating a suitable image for the user, and then returns the generated image and a matching image to the terminal device of the user as image retrieval results, facilitating the user to select an image. After obtaining the image description information of the expected image from the user, the server generates an initial generated image based on the expected content description and the expected image style in the image description information, and extracts the semantic of the expected content description to obtain key image information corresponding to the expected content description. Then, the server performs image content recognition on the initial generated image to obtain content recognition data, and performs style recognition on the initial generated image to obtain image recognition style. It is determined whether the image recognition style is consistent with the expected image style, and whether the content recognition data matches the key image information. If the image recognition style is consistent with the expected image style, and the content recognition data matches the key image information, the initial generated image is output as a target retrieval image corresponding to the expected image. In this embodiment, when the user performs expected image retrieval, the initial generated image is generated based on the expected image style and the expected content description input by the user, and when it is determined that the initial generated image meets the image description information of the expected image from the user, the generated initial generated image is output as an image retrieval result. This not only enables the user to retrieve more satisfactory image retrieval data and improves the image retrieval effect, but also improves the flexibility of image data acquisition.

[0058] The terminal device can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and the like. The server can be implemented by a single server or a server cluster composed of multiple servers.

[0059] In an embodiment, as shown in Figure 2 a data processing method is provided. Taking a server in Figure 1 as an example, the method comprises the following steps:

[0060] S10: Obtain image description information of a desired image from a user, the image description information comprising a desired image style and a desired content description.

[0061] When the user needs to perform image search, the user needs to input a keyword (text information or voice information) of the desired image through the terminal device, so as to perform image search according to the image description information, and then select a suitable image according to the image search result. In the traditional search method, the image search result returned by the server is usually the image data already existing in the database, which limits the flexibility of image data acquisition, and the image search result is not ideal, and it is difficult to find an image satisfying the user.

[0062] To solve the foregoing problem, in the embodiment, when the user has an image search requirement, for example, when the user edits H5, a poster and an activity page on a preset editing platform, and needs to add or replace an image picture satisfying the user, the image search can be performed by using the existing search engine on the preset editing platform, so as to search for an image satisfying the user. In the search process, the user can input the image description information of the desired image to the search engine of the preset editing platform through the terminal device. The server of the preset editing platform obtains the image description information of the desired image from the user, and generates an image according to the image description information (the image description information comprising a desired image style and a desired content description of the desired image from the user). At the same time, the server also performs database image matching according to the traditional method, that is, extracts a key field in the image description information, matches the key field with a label of an existing image (a historical image) in the database, and then obtains the matched image, so as to return the generated image and the matched image to the user as the image search result, thereby improving the flexibility of image data acquisition and improving the image search effect.

[0063] In the embodiment, the image description information is a content description of an image (i.e., a desired image) to be generated from the user, comprising a desired image style and a desired content description of the desired image from the user. The desired content description can be an image element (a constituent element of the image), an image background (which can include a background pattern, a background texture, a background color and the like), a color and a position of the image element and the like. The image description information is natural language information, which can be text information or voice information.

[0064] S20: generating an initial generated image based on the expected content description and the expected image style, and performing semantic extraction on the expected content description to obtain key image information corresponding to the expected content description.

[0065] After obtaining the image description information of the expected image by the user, that is, obtaining the expected image style and the expected content description of the expected image by the user, the server needs to generate an image based on the expected content description and the expected image style to obtain an initial generated image, which not only considers the image content, but also considers the image style for the marketing of the generation effect, so that the initial generated image maximizes the user's requirements. In this embodiment, based on the expected content description and the expected image style, a pre-trained target image generation model can be used to perform an image generation task to obtain an initial generated image. For example, the target image generation model can be a model based on a variational autoencoder. The expected content description is encoded to obtain a feature vector of the expected content description, and the expected image style is encoded to obtain a feature vector of the expected image style. Then, the target image generation model is used to perform pixel mapping in a high-dimensional space on the feature vectors of the expected content description and the expected image style to obtain the initial generated image. In other embodiments, the pre-set image generation model can also be other models for generating images from text, which will not be described here.

[0066] At the same time, the server also needs to perform semantic extraction on the expected content description to obtain key image information corresponding to the expected content description. The key image information includes the composition elements, the image background, and the color and position of the composition elements of the expected image, and the like. Through the rich key image information, more details of the expected image can be obtained, so that the subsequent model can better mine the granularity of the image and generate a better image.

[0067] In the semantic extraction of the expected content description, the corresponding key image information of the expected content description is obtained, including: obtaining the editing data of the user on the editing interface of the preset editing platform, and obtaining the user portrait and the historical behavior data of the user on the preset editing platform; in the process of semantic extraction of the expected content description by the server, semantic association is performed based on the knowledge graph, the editing data of the user on the editing interface, the user portrait and the historical behavior data of the user, and the like, to obtain the corresponding key image information of the expected content description. That is, in the process of semantic extraction of the expected content description by the server, if it is detected that there is editing data (such as a poster or H5 work being edited) of the user on the preset editing platform, the editing data of the user on the editing interface can be obtained, and then the server can perform semantic association based on the knowledge graph, the editing data of the user on the editing interface, the user portrait and the historical behavior data of the user, and the like, to obtain more perfect key image information. In the process of semantic extraction of the expected content description by the server, semantic association is performed based on the knowledge graph, which can intelligently expand the expected content description from the perspective of the graph, thereby enriching the user's intention, increasing the accuracy of the expected content description, and further improving the accuracy of the key image information. In the semantic extraction, the user portrait and the historical behavior data of the user are added to the knowledge graph to perform semantic association, which can further expand the expected content description, so that the recognized user intention is closer to the user's behavior preference, thereby improving the accuracy of the key image information and improving the image generation effect. In the semantic extraction, the editing data of the user on the editing interface is added to the knowledge graph to perform semantic association, which can further expand the expected content description, and make the recognized user intention more consistent with the current editing needs of the user, thereby making the subsequently generated image more consistent with the user's needs.

[0068] S30: image content recognition is performed on the initial generated image to obtain content recognition data, and style recognition is performed on the initial generated image to obtain an image recognition style.

[0069] After the image generation based on the expected content description and the expected image style obtains the initial generated image, the server needs to perform image content recognition on the initial generated image to obtain content recognition data; at the same time, the server also needs to perform style recognition on the initial generated image to obtain an image recognition style.

[0070] In this embodiment, a pre-trained style recognition model can be used to perform style recognition on the initial generated image to quickly obtain the image recognition style of the initial generated image while ensuring recognition accuracy. A pre-trained content recognition model can be used to perform recognition on the image content such as the composition elements, element colors, element positions, and backgrounds of the initial generated image to quickly obtain accurate and diversified content recognition data.

[0071] S40: Determine whether the image recognition style is consistent with the expected image style, and determine whether the content recognition data matches the key image information.

[0072] After identifying the content recognition data and the image recognition style of the initial generated image, the server also needs to determine whether the image recognition style is consistent with the expected image style, and determine whether the content recognition data matches the key image information, to confirm whether the initial generated image meets the image description information input by the user, that is, to confirm whether the initial generated image meets the user description requirements.

[0073] S50: If the image recognition style is consistent with the expected image style, and the content recognition data matches the key image information, output the initial generated image as the target search image corresponding to the expected image.

[0074] After determining whether the image recognition style is consistent with the expected image style, and determining whether the content recognition data matches the key image information, if it is determined that the image recognition style is consistent with the expected image style, and it is determined that the content recognition data matches the key image information, it means that the image style of the initial generated image is consistent with the image style required in the image description information input by the user, and the image content of the initial generated image is the image content required in the image description information input by the user, that is, the initial generated image meets the user description requirements, then output the initial generated image as the target search image corresponding to the expected image, so that the target search image completely meets the user description requirements, is closer to the expected image of the user, and improves the image generation effect of the target search image.

[0075] At the same time, the server will also perform database image matching in a traditional manner, that is, extract the key field in the image description information, match the key field with the labels of the existing images (historical images) in the database, and then obtain the matched images. In order to subsequently return the generated images and the matched images as image search results to the user, so as to improve the flexibility of image data acquisition and improve the image search effect. After outputting the initial generated image as the target search image corresponding to the expected image, the server returns the initial generated image and the images matched in the database to the user as image search results, so as to improve the flexibility of image data acquisition and improve the image search effect.

[0076] If it is determined that the image recognition style of the initial generated image is inconsistent with the expected image style, indicating that the image style of the initial generated image is inconsistent with the image style required in the image description information input by the user, the initial generated image does not meet the user description requirement, the effect of the initial generated image is poor, the server needs to generate a new initial generated image according to the image description information input by the user, and the initial generated image is output as the target search image corresponding to the expected image after the new initial generated image meets the user description requirement. Or, if it is determined that the content recognition data of the initial generated image does not match the key image information, for example, the image elements of the initial generated image do not match (missing image elements described in the expected content, adding image elements not included in the expected content), the position or color of the same element does not match, the image background does not match, etc., indicating that the image content of the initial generated image is not the image content required in the image description information input by the user, the effect of the initial generated image is poor, the server needs to generate a new initial generated image according to the image description information input by the user, and the initial generated image is output as the target search image corresponding to the expected image after the new initial generated image meets the user description requirement. When it is determined that the image recognition style is inconsistent with the expected image style, or the content recognition data does not match the key image information, the server generates a new image again, which can ensure the image generation effect.

[0077] In the embodiment, when the user performs image search, the image description information of the expected image is obtained, the image description information includes the expected image style and the expected content description, image generation is performed based on the expected content description and the expected image style to obtain an initial generated image, semantic extraction is performed on the expected content description to obtain the key image information corresponding to the expected content description, image content recognition is performed on the initial generated image to obtain content recognition data, and style recognition is performed on the initial generated image to obtain an image recognition style. It is determined whether the image recognition style is consistent with the expected image style, and whether the content recognition data matches the key image information. If the image recognition style is consistent with the expected image style, and the content recognition data matches the key image information, the initial generated image is output as the target search image corresponding to the expected image. In the present application, when the user performs expected image search, the initial generated image is generated based on the expected image style and the expected content description input by the user, and the generated initial generated image is output as the image search result when it is determined that the initial generated image meets the image description information of the expected image input by the user. Not only can the user retrieve more satisfactory image search data to improve the image search effect, but also the flexibility of image data acquisition can be improved.

[0078] In an embodiment, as Figure 3As shown, in step S20, i.e. based on the desired content description and the desired image style, the image generation is performed to obtain an initial generated image, which specifically includes the following steps:

[0079] S21: Obtain a target image generation model obtained by deep learning of a generative adversarial network based on a model sample data set.

[0080] After obtaining the image description information of the desired image by the user, the server also needs to obtain a pre-trained target image generation model, so as to call the target image generation model to generate an image based on the desired image style and the desired content description. The target image generation model is a target image generation model obtained by deep learning of a generative adversarial network based on a model sample data set. The model sample data set includes a plurality of historical image samples, and each historical image sample in the model sample data set corresponds to a sample content description vector and a sample style vector.

[0081] In the model training process, the generative adversarial network is trained by deep learning based on the sample content description vector and the sample style vector of the plurality of historical image samples, and the target image generation model is obtained. The generative adversarial network can be a general neural network structure including a generation module and a discrimination module; in the model training process, the sample content description vector and the sample style vector of a certain historical image sample are taken as the input of the generation module, so that the generation module generates an image based on the sample content description vector and the sample style vector; the discrimination module is used to judge the authenticity of the image generated by the generation module, and a loss function is calculated based on the historical image sample, until the loss function converges and the generation module is output as the target image generation model.

[0082] S22: Perform semantic feature extraction on the desired content description to obtain an image description vector, and perform semantic feature extraction on the desired image style to obtain a desired image style vector.

[0083] After obtaining the pre-trained target image generation model, the semantic feature extraction is performed on the desired content description to obtain an image description vector, and the semantic feature extraction is performed on the desired image style to obtain a desired image style vector.

[0084] In the process of performing semantic feature extraction on the expected content description and / or the expected image style, semantic association can be performed based on the knowledge graph, the editing data of the user on the editing interface, the user portrait, and the user historical behavior data, to obtain the expected content description and / or the expected image style after semantic association, and then feature encoding is performed on the expected content description and / or the expected image style after semantic association, to obtain the image description vector and the expected image style vector. In the process of performing semantic feature extraction on the expected content description by the server, semantic association based on the knowledge graph can intelligently expand the expected content description from the perspective of the graph, thereby enriching the user's intention, increasing the diversity of the expected content description, and further improving the accuracy of the image description vector and the expected image style vector, so as to improve the generation effect of the subsequent image. In the process of semantic feature extraction, the user portrait and the user historical behavior data are added to the knowledge graph to perform semantic association, which can further expand the expected content description, so that the recognized user intention is closer to the user's behavior preference, further improving the accuracy of the corresponding vector, and further improving the image generation effect. In the process of semantic feature extraction, the editing data of the user on the editing interface is added to the knowledge graph to perform semantic association, which can further expand the expected content description, and the recognized user intention is more in line with the current editing needs of the user, and the subsequent generated image is more in line with the user's needs.

[0085] S23: input the expected image style vector and the image description vector into the target image generation model to perform image generation, and obtain the image output by the target image generation model as an initial generated image.

[0086] After obtaining the image description vector and the expected image style vector, the expected image style vector and the image description vector are input into the target image generation model to perform image generation, and the image output by the target image generation model is obtained as an initial generated image.

[0087] In this embodiment, the target image generation model obtained by performing deep learning on the generative adversarial network based on the model sample data set is obtained, then the semantic feature extraction is performed on the expected content description to obtain the image description vector, and the semantic feature extraction is performed on the expected image style to obtain the expected image style vector, and then the expected image style vector and the image description vector are input into the target image generation model to perform image generation, and the image output by the target image generation model is obtained as an initial generated image. The specific steps of generating an initial generated image based on an expected content description and an expected image style are determined, a high-precision target image generation model is obtained by training a generative adversarial network based on a plurality of historical image sample content description vectors and sample style vectors, and the model is directly called for image generation in subsequent steps, which is simple, fast, and has high image quality.

[0088] In a preferred embodiment, the sample content description vector of the historical image sample includes a semantic vector of the sample constituent element, a position vector of the sample constituent element, and a color vector. The generative adversarial network includes a generative module, a style transfer network, and a discriminative module, wherein the style transfer network and the discriminative module are used to assist the generative module to generate the image, so as to reduce the difference between the generated image and the historical image sample in terms of the expected image style, the position of the image element, and the color. During the model training process, the semantic vector of the sample constituent element, the position vector of the sample constituent element, and the color vector of the sample constituent element in the historical image sample are input into the generative module to generate the image, and then the generated image is introduced into the style transfer network for rendering based on the sample style vector, so as to generate the image style information for the generative module. Finally, the discriminative module is used to measure the difference between the generated image and the historical image sample, so as to guide the generative module to generate an image that is closer to the historical image sample in terms of the image style, the image constituent element, the position, and the color. After the loss function converges, the converged generator and the style transfer network are output as the target image generation model, and the structure of the target image generation model enables the target image generation model to mine image information in a fine-grained manner and generate an image, which has good performance in terms of the visual quality and the image granularity. Correspondingly, during the process of performing semantic feature extraction on the expected content description to obtain the image description vector, the text vector of the expected element, the position vector of the expected element, and the color vector of the expected element need to be extracted from the expected content description, i.e., the image description vector can include the text vector of the expected element, the position vector of the expected element, and the color vector of the expected element. Then, after the target image generation model is trained, the expected image style vector, the semantic vector of the expected element, the position vector of the expected element, and the color vector of the expected element are input into the target image generation model to generate an image, and the image output by the target image generation model is obtained as the initial generated image corresponding to the expected image, so that an image with good performance in terms of the visual quality and the image granularity can be obtained, and the generation effect of the initial generated image is improved.

[0089] In an embodiment, the key image information includes a plurality of expected elements of the expected image. As shown in FIG. 4, in step S40, i.e., determining whether the content recognition data matches the key image information, the following steps are specifically included: Figure 4

[0090] S41: performing constituent element recognition on the content recognition data to obtain a plurality of constituent elements of the initial generated image;

[0091] S42: one-to-one matching the plurality of expected elements with the plurality of constituent elements to obtain a matching result;

[0092] S43: if the element matching result is complete matching, it is determined that the content recognition data matches the key image information;

[0093] ​S44: If the element matching result is not complete matching, it is determined that the content recognition data does not match the key image information.

[0094] In this embodiment, the key image information includes a plurality of expected elements of the expected image. After the content recognition data is obtained by performing image content recognition on the initial generated image, element composition recognition is performed on the content recognition data to obtain a plurality of composition elements of the initial generated image. Then, the plurality of expected elements and the plurality of composition elements are matched one by one to obtain a matching result. If the element matching result is complete matching, it indicates that the composition element set of the initial generated image completely coincides with the expected element set of the expected image, and the composition element information of the initial generated image is consistent with the user expected image. Therefore, it is determined that the content recognition data matches the key image information. If the element matching result is not complete matching, it indicates that the composition element set of the initial generated image does not completely coincide with the expected element set of the expected image, and the composition element information of the initial generated image is not consistent with the user expected image. Therefore, it is determined that the content recognition data does not match the key image information.

[0095] For example, the plurality of expected elements include element A, element B and element C, and the plurality of composition elements of the initial generated image are element A, element B and element D, i.e., the composition element set of the initial generated image does not completely coincide with the expected element set of the expected image. Therefore, it is determined that the content recognition data does not match the key image information. If the plurality of composition elements of the initial generated image are also element A, element B and element C, it is determined that the content recognition data matches the key image information.

[0096] In this embodiment, the plurality of composition elements of the initial generated image are obtained by performing element composition recognition on the content recognition data. Then, the plurality of expected elements and the plurality of composition elements are matched one by one to obtain a matching result. If the element matching result is complete matching, it is determined that the content recognition data matches the key image information. If the element matching result is not complete matching, it is determined that the content recognition data does not match the key image information. The specific steps of determining whether the content recognition data matches the key image information are refined. When performing image data matching, the composition elements of the images are matched one by one, which ensures the accuracy of the matching result and provides a judgment basis for subsequent data processing.

[0097] In other embodiments, the key image information can also include one or more of the position and color of each expected element, and the background and other information of the expected image. Correspondingly, the content recognition data of the initial generated image also includes one or more of the position and color of each composition element in the initial generated image, and the background and other information of the initial generated image. When matching the initial generated image and the user expected image, not only the image elements themselves are matched, but also other information is matched to increase the information matching granularity, thereby increasing the accuracy of the matching result.

[0098] In an embodiment, the key image information further includes the position and color of each desired element. In step S42, the plurality of desired elements are matched with the plurality of constituent elements one by one to obtain a matching result, which includes the following steps:

[0099] S421: Determine whether the plurality of constituent elements can be matched with the plurality of desired elements one by one;

[0100] S422: If the plurality of constituent elements can be matched with the plurality of desired elements one by one, determine the position and color of each constituent element based on the content recognition data;

[0101] S423: Determine whether the position and color of each constituent element match the position and color of the corresponding desired element;

[0102] S424: If the position and color of each constituent element match the position and color of the corresponding desired element, determine that the matching result is a complete match;

[0103] S425: If the position or color of any constituent element does not match the position or color of the corresponding desired element, determine that the matching result is not a complete match.

[0104] After the constituent element recognition of the content recognition data is performed to obtain the plurality of constituent elements of the initial generated image, it is determined whether the plurality of constituent elements can be matched with the plurality of desired elements one by one. If the plurality of constituent elements cannot be matched with the plurality of desired elements one by one, i.e., the constituent element set of the initial generated image does not completely coincide with the desired element set of the desired image, it indicates that the constituent elements of the initial generated image do not match the desired elements of the user, and it is determined that the content recognition data does not match the key image information.

[0105] If the plurality of constituent elements can be matched with the plurality of desired elements one by one, i.e., the constituent element set of the initial generated image does not completely coincide with the desired element set of the desired image, it indicates that the constituent elements of the initial generated image match the desired elements of the user. At this time, further judgment of other information is required. Since the key image information in this embodiment includes the plurality of desired elements and the position and color of each desired element, it is also necessary to determine the position and color of each constituent element based on the content recognition data. Then, it is determined whether the position and color of each constituent element match the position and color of the corresponding desired element. If the position and color of each constituent element match the position and color of the corresponding desired element, it is determined that the matching result is a complete match. If the position or color of any constituent element does not match the position or color of the corresponding desired element, it is determined that the matching result is not a complete match.

[0106] For example, the plurality of desired elements includes element A and element B, and the plurality of constituent elements of the initially generated image is element A and element B. If the color and position of element A in the desired image are the same as those of element A in the initially generated image, and the color and position of element B in the desired image are the same as those of element B in the initially generated image, it is determined that the matching result is complete matching. If the color or position of element A (or B) in the desired image is different from that of element A (or B) in the initially generated image, or the color and position of element A in the desired image are the same as those of element A in the initially generated image, but the color or position of element B in the desired image is different from that of element B in the initially generated image, it is determined that the matching result is not complete matching.

[0107] In this embodiment, the key image information further includes the position and color of each desired element. Whether the plurality of constituent elements and the plurality of desired elements can be matched one by one is determined. If the plurality of constituent elements and the plurality of desired elements can be matched one by one, the position and color of each constituent element are determined based on the content recognition data. Whether the position and color of each constituent element match the position and color of the corresponding desired element is determined. If the position and color of each constituent element match the position and color of the corresponding desired element, it is determined that the matching result is complete matching. If the plurality of constituent elements and the plurality of desired elements cannot be matched one by one, or the position or color of any constituent element does not match the position or color of the corresponding desired element, it is determined that the matching result is not complete matching. The specific steps of matching the plurality of desired elements and the plurality of constituent elements one by one to obtain the matching result are clarified. When matching the elements, not only the elements themselves but also the colors and positions of the elements are matched. The granularity of information matching is increased, thereby increasing the accuracy of the matching result.

[0108] In addition, only when the position and color of each constituent element match the position and color of the corresponding desired element, it is determined that the matching result is complete matching. The accuracy of the matching result is improved, thereby improving the accuracy of the subsequent output target search image, and further improving the user search effect. In other embodiments, when the position and color of the plurality of constituent elements in the initially generated image match the position and color of the corresponding desired element, when the proportion of the matching constituent elements is greater than a preset proportion (such as 80%), it is determined that the matching result is complete matching. This way reduces the requirement for image accuracy. On the basis that the initially generated image basically meets the user description, the target search image meeting the requirement can be quickly obtained, the number and diversity of the target search image are increased, and more image options are provided for the user.

[0109] In an embodiment, in step S423, whether the position and color of each constituent element match the position and color of the corresponding desired element is determined, specifically including the following steps:

[0110] S4231: Calculate the distance between the position of each component element and the position of the corresponding expected element, to obtain the position distance of each component element.

[0111] In this embodiment, the key image information includes multiple expected elements, the position and color of each expected element; when performing component element recognition on the content recognition data, multiple component elements of the initial generated image and the position and color of each component element are obtained. After determining that the multiple component elements and the multiple expected elements can be one-to-one corresponding matching, the distance between the position of each component element and the position of the corresponding expected element needs to be calculated to obtain the position distance of each component element. The greater the position distance is, the greater the deviation between the position of the component element and the position of the corresponding expected element is; the smaller the position distance is, the smaller the deviation between the position of the component element and the position of the corresponding expected element is, and the more similar the element positions are.

[0112] In the calculation of the position distance, the Euclidean distance algorithm can be used to calculate the shortest distance between two positions, which is simple and intuitive. In other embodiments, other distance calculation methods can also be used, such as the cosine distance algorithm, the standard Euclidean distance algorithm, etc.

[0113] S4232: Calculate the similarity between the color of each component element and the color of the corresponding expected element to obtain the color difference of each component element.

[0114] After determining that the multiple component elements and the multiple expected elements can be one-to-one corresponding matching, the similarity between the color of each component element and the color of the corresponding expected element also needs to be calculated to obtain the color difference of each component element. The algorithm of the color difference can be a three-dimensional space-based color difference calculation method, and the implementation process is usually as follows: converting the RGB color space to the normalized hsv color space, converting the three-dimensional coordinate points of the hsv color space, then calculating the distance between the three-dimensional coordinate points of the two colors, and taking the distance between the three-dimensional coordinate points as the color difference of the two colors; the more similar the two colors are, the closer the coordinate distance of the two colors is to 0, and vice versa, the farther the two colors are, the closer the coordinate distance is to 1. Wherein, HSV (Hue, Saturation, Value) is a color space created according to the intuitive characteristics of color, also known as a hexagonal cone model, in which the parameters of color are hue (H), saturation (S) and brightness (V).

[0115] S4233: Determine whether the position distance of each component element is less than a preset distance, and determine whether the color difference of each component element is less than a preset color difference value.

[0116] The smaller the position distance between the constituent elements is, the closer the position of the constituent elements is to the position expected by the user, and the higher the image accuracy is; the smaller the color difference is, the closer the color between the constituent elements is, and the closer the color of the constituent elements is to the color expected by the user, and the higher the image accuracy is. After obtaining the position distance of each constituent element and the color difference of each constituent element, it is determined whether the position distance of each constituent element is less than a preset distance, and whether the color difference of each constituent element is less than a preset color similarity.

[0117] S4234: If the position distance of each constituent element is less than the preset distance, and the color difference of each constituent element is less than the preset color difference value, it is determined that the position and color of each constituent element match the position and color of the corresponding expected element.

[0118] After determining whether the position distance of each constituent element is less than the preset distance, and whether the color difference of each constituent element is less than the preset color difference value, if the position distance of each constituent element is less than the preset distance, it indicates that the position of each constituent element in the initially generated image is relatively close to the position expected by the user, which can be ignored, and the color difference of each constituent element is less than the preset color difference value, which indicates that the color of each constituent element in the initially generated image is relatively close to the color expected by the user, and the human eye cannot perceive the color difference, so it is determined that the position and color of each constituent element match the position and color of the corresponding expected element, that is, it can be determined that the matching result is complete matching.

[0119] If the position distance of any constituent element is greater than or equal to the preset distance, it indicates that the position of the constituent element in the initially generated image is relatively far from the position expected by the user, and the position of the constituent element does not match the position of the corresponding expected element, so it is determined that the matching result is not complete matching; if the color difference of any constituent element is greater than or equal to the preset color difference value, it indicates that the color of the constituent element in the initially generated image is relatively far from the color expected by the user, and the color of the constituent element does not match the color of the corresponding expected element, so it is determined that the matching result is not complete matching.

[0120] To ensure the image accuracy of the generated small-size picture, the preset distance can be a distance less than 1 cm, such as 0.9 cm, 0.8 cm, or 0.5 cm. To reduce the error of the human eye caused by different colors, the preset color difference value needs to be less than 0.5; to reduce the color difference and make the color of the constituent element closer to the color expected by the user, thereby improving the image accuracy, the preset color difference value can be 0.3.

[0121] In this embodiment, the position distance of each component element is obtained by calculating the distance between the position of each component element and the position of the corresponding expected element, and the color difference of each component element is obtained by calculating the similarity between the color of each component element and the color of the corresponding expected element. Then, it is determined whether the position distance of each component element is less than the preset distance, and whether the color difference of each component element is less than the preset color difference value. If the position distance of each component element is less than the preset distance, and the color difference of each component element is less than the preset color difference value, it is determined that the position and color of each component element match the position and color of the corresponding expected element. The specific implementation of determining whether the position and color of each component element match the position and color of the corresponding expected element is clarified, which provides accurate data basis for subsequent judgment.

[0122] In other embodiments, the key image information can also include one or more of the position and color of each expected element, and the background information of the expected image. Correspondingly, the content recognition data of the initial generated image also includes one or more of the position and color of each component element in the initial generated image, and the background information of the initial generated image. When performing one-to-one matching of elements, other information also needs to be matched to increase the granularity of information matching, thereby increasing the accuracy of the matching result.

[0123] In an embodiment, taking the case that the key image information also includes the position and color of each expected element as an example. In step S42, the plurality of expected elements and the plurality of component elements are matched one by one to obtain a matching result, which specifically includes the following steps:

[0124] In an embodiment, taking the case that the key image information also includes the expected background description (background of the expected image) as an example. As shown in Figure 5 After step S42, i.e., after the matching result of matching the plurality of expected elements and the plurality of component elements one by one is obtained, before determining whether the content recognition data matches the key image information, the method further includes the following steps:

[0125] S45: If the matching result is complete matching, perform background recognition on the content recognition data to obtain the image background description of the initial generated image.

[0126] After performing component element recognition on the content recognition data to obtain the plurality of component elements of the initial generated image, it is determined whether the plurality of component elements and the plurality of expected elements can be matched one by one. If the plurality of component elements and the plurality of expected elements cannot be matched one by one, it is determined that the content recognition data does not match the key image information.

[0127] If the plurality of constituent elements can be matched with the plurality of expected elements one by one, it indicates that the constituent element information of the initial generated image is consistent with the user expected element information, at this time, further judgment of other information is needed. Since the key image information in the embodiment includes a plurality of expected elements and expected background description, i.e. matching each constituent element of the initial generated image, it is also necessary to determine the image background (denoted as image background description) of the initial generated image based on the content recognition data, so as to subsequently perform background matching and determine whether the content recognition data matches the key image information based on the background matching result, i.e. determine whether the initial generated image meets the user description requirement.

[0128] If the plurality of expected elements and the plurality of constituent elements are matched one by one to obtain a matching result of complete match, it is also necessary to perform background recognition on the content recognition data to obtain the image background description of the initial generated image. The image background description is a semantic description content of the background of the initial generated image.

[0129] S46: Determine the background similarity of the expected background description and the image background description.

[0130] Then, the background similarity of the expected background description and the image background description is determined, i.e. the similarity between the image background of the initial generated image and the image background expected by the user is determined, i.e. the similarity of the background of the initial generated image and the background of the expected image is calculated based on the expected background description and the image background description.

[0131] For example, the expected background description and the image background description can be respectively subjected to semantic extraction to obtain a plurality of expected background keywords and a plurality of image background keywords; then, the plurality of expected background keywords are respectively matched with the plurality of image background keywords to obtain a keyword matching rate, and the keyword matching rate is taken as the background similarity.

[0132] S47: If the background similarity is greater than or equal to a preset background similarity, it is determined that the content recognition data matches the key image information.

[0133] S48: If the background similarity is less than the preset background similarity, it is determined that the content recognition data does not match the key image information.

[0134] After determining the background similarity of the expected background description and the image background description, if the background similarity is greater than or equal to a preset background similarity, it indicates that the image background of the initial generated image is relatively close to the image background expected by the user, at this time, it can be determined that the content recognition data matches the key image information; if the background similarity is less than the preset background similarity, it indicates that the image background of the initial generated image is greatly different from the image background expected by the user, at this time, it can be determined that the content recognition data does not match the key image information.

[0135] In the embodiment, the key image information further includes an expected background description. After the plurality of expected elements are matched with the plurality of constituent elements one by one to obtain a matching result, if the matching result is a complete match, the content recognition data is subjected to background recognition to obtain an image background description of the initial generated image; a background similarity between the expected background description and the image background description is determined; if the background similarity is greater than or equal to a preset background similarity, it is determined that the content recognition data matches the key image information; if the background similarity is less than the preset background similarity, it is determined that the content recognition data does not match the key image information. The specific steps of determining whether the content recognition data matches the key image information are clarified. On the basis of judging the matching condition of the image constituent elements, the matching condition of the image background is also considered, the image effect of the initial generated image is investigated in multiple dimensions, and the accuracy of the judgment is improved.

[0136] In an embodiment, the expected background description includes an expected background color and an expected background pattern, and the image background description includes a recognized background color and a recognized background pattern. In step S46, the background similarity between the expected background description and the image background description is determined, specifically including the following steps:

[0137] S461: Calculate the background color similarity between the expected background color and the recognized background color, and calculate the pattern similarity between the expected background pattern and the recognized background pattern.

[0138] In the embodiment, the expected background description includes an expected background color and an expected background pattern. Correspondingly, the background recognition of the content recognition data to obtain the image background description of the initial generated image also includes a recognized background color and a recognized background pattern. When the background similarity between the expected background description and the image background description is determined, the background color similarity between the expected background color and the recognized background color needs to be calculated, and the pattern similarity between the expected background pattern and the recognized background pattern needs to be calculated.

[0139] Wherein, before calculating the background color similarity between the expected background color and the recognized background color, a color relationship graph constructed in advance needs to be obtained, and then the background color similarity between the expected background color and the recognized background color is determined based on the color relationship graph. The color relationship graph is composed of a plurality of nodes and edges between the nodes. The nodes in the color relationship graph represent colors, and the edges between two nodes represent the similarity of the two colors. The more similar the colors are, the greater the color similarity is, and the shorter the edge is. For example, the similarity of magenta and red is relatively large, and the edge between them is relatively short; the similarity of green and red is relatively small, and the edge between them is very small and close to 0. Therefore, the color relationship graph can be used to quickly and accurately determine the background color similarity between the expected background color and the recognized background color.

[0140] Wherein, before calculating the pattern similarity between the expected background pattern and the identified background pattern, a pre-constructed pattern relationship graph is needed to be obtained, and then the pattern similarity between the expected background pattern and the identified background pattern is determined based on the pattern relationship graph. The pattern relationship graph is composed of multiple nodes and edges between the nodes. The nodes in the pattern relationship graph represent patterns (including textures, graphics and symbols). The edges between two nodes represent the similarity of the two patterns. The more similar the patterns are, the greater the pattern similarity is, and the shorter the edge is. For example, the similarity of fine corrugated pattern and coarse corrugated pattern is relatively large, and the edge between them is relatively short. The similarity of fine corrugated pattern and musical note is relatively small, and the edge between them is very small and close to 0. Therefore, the pattern similarity between the expected background pattern and the identified background pattern can be quickly and accurately determined through the pattern relationship graph.

[0141] S462: Calculate the background similarity based on the background color similarity and the pattern similarity.

[0142] After the background color similarity and the pattern similarity are calculated, the background similarity needs to be calculated based on the background color similarity and the pattern similarity. For example, the background color similarity and the pattern similarity can be averaged to obtain the average value as the background similarity, which is simple and accurate. Or, the sum of the background color similarity and the pattern similarity is taken as the background similarity.

[0143] In other embodiments, different weights can be set for the background color similarity and the pattern similarity, and then the background color similarity and the pattern similarity are weighted and averaged according to the weight of the background color similarity and the weight of the pattern similarity to obtain the background similarity. By calculating the background similarity in the weighted average manner, the importance of color or pattern can be emphasized according to the requirements, so as to obtain a more accurate background similarity. Wherein, the weights can be dynamically determined according to the image description information of the user.

[0144] For example, if the image (expected image) expected to be generated by the user is determined to be a black and white image according to the image description information of the user, it means that the user does not emphasize the color of the image. At this time, it can be determined that the weight of the pattern similarity is greater than the weight of the color similarity, and the texture pattern of the image is more important. If the image (expected image) expected to be generated by the user is determined to be a color image according to the image description information of the user, it means that the user emphasizes the color of the image. At this time, it can be determined that the weight of the pattern similarity is less than the weight of the color similarity, and the color of the image is more important.

[0145] In this embodiment, the expected background description includes an expected background color and an expected background pattern, the image background description includes an identified background color and an identified background pattern, the background similarity between the expected background description and the image background description is determined by calculating a background color similarity between the expected background color and the identified background color and a pattern similarity between the expected background pattern and the identified background pattern, and the background similarity is calculated based on the background color similarity and the pattern similarity, so that the specific steps of determining the background similarity between the expected background description and the image background description are determined, the background similarity can be calculated more accurately according to the background color similarity and the pattern similarity, and a basis is provided for subsequent judgment.

[0146] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0147] In an embodiment, a data processing apparatus is provided, which corresponds to the data processing method in the above embodiment. As shown in the figure, the data processing apparatus includes an acquisition module 601, a generation module 602, an identification module 603, a determination module 604 and an output module 605. The functions of each functional module are described in detail as follows. Figure 6

[0148] The acquisition module 601 is configured to acquire image description information of an expected image from a user, the image description information including an expected image style and an expected content description.

[0149] The generation module 602 is configured to generate an image based on the expected content description and the expected image style to obtain an initial generated image, and perform semantic extraction on the expected content description to obtain key image information corresponding to the expected content description.

[0150] The identification module 603 is configured to perform image content identification on the initial generated image to obtain content identification data, and perform style identification on the initial generated image to obtain an image identification style.

[0151] The determination module 604 is configured to determine whether the image identification style is consistent with the expected image style, and determine whether the content identification data matches the key image information.

[0152] The output module 605 is configured to output the initial generated image as a target retrieval image corresponding to the expected image if the image identification style is consistent with the expected image style and the content identification data matches the key image information.

[0153] Optionally, the key image information includes a plurality of expected elements of the expected image, and the determination module 604 is specifically configured to:

[0154] perform component element identification on the content identification data to obtain a plurality of component elements of the initial generated image.​

[0155] The plurality of expected elements are matched with the plurality of component elements one by one to obtain a matching result;

[0156] If the element matching result is complete matching, it is determined that the content recognition data matches the key image information;

[0157] If the element matching result is not complete matching, it is determined that the content recognition data does not match the key image information.

[0158] Optionally, the key image information further includes a position and a color of each expected element, and the determining module 604 is specifically further used for:

[0159] determining whether the plurality of component elements can be matched with the plurality of expected elements one by one;

[0160] If the plurality of component elements can be matched with the plurality of expected elements one by one, the position and the color of each component element are determined based on the content recognition data;

[0161] It is determined whether the position and the color of each component element match the position and the color of the corresponding expected element;

[0162] If the position and the color of each component element match the position and the color of the corresponding expected element, it is determined that the matching result is complete matching;

[0163] If the plurality of component elements cannot be matched with the plurality of expected elements one by one, or the position or the color of any component element does not match the position or the color of the corresponding expected element, it is determined that the matching result is not complete matching.

[0164] Optionally, the determining module 604 is specifically further used for:

[0165] calculating a distance between the position of each component element and the position of the corresponding expected element to obtain a position distance of each component element;

[0166] calculating a similarity between the color of each component element and the color of the corresponding expected element to obtain a color difference of each component element;

[0167] It is determined whether the position distance of each component element is less than a preset distance, and whether the color difference of each component element is greater than a preset color difference value;

[0168] If the position distance of each component element is less than the preset distance, and the color difference of each component element is greater than the preset color difference value, it is determined that the position and the color of each component element match the position and the color of the corresponding expected element.

[0169] Optionally, the key image information further comprises an expected background description, after matching the plurality of expected elements with the plurality of constituent elements and obtaining the matching result, the determining module 604 is specifically further configured to:

[0170] if the matching result is a complete match, performing background recognition on the content recognition data to obtain an image background description of the initial generated image;

[0171] determining a background similarity between the expected background description and the image background description;

[0172] if the background similarity is greater than or equal to a preset background similarity, determining that the content recognition data matches the key image information;

[0173] if the background similarity is less than the preset background similarity, determining that the content recognition data does not match the key image information.

[0174] Optionally, the expected background description comprises an expected background color and an expected background pattern, and the image background description comprises a recognized background color and a recognized background pattern, and the determining module 604 is specifically further configured to:

[0175] calculating a background color difference between the expected background color and the recognized background color, and calculating a pattern similarity between the expected background pattern and the recognized background pattern;

[0176] calculating the background similarity based on the background color difference and the pattern similarity.

[0177] Optionally, the generating module 602 is specifically configured to:

[0178] obtain a target image generation model obtained by deep learning of a generative adversarial network based on a model sample data set, each historical image sample in the model sample data set corresponding to a sample content description vector and a sample style vector, the generative adversarial network being trained based on the sample content description vectors and the sample style vectors of the plurality of historical image samples to obtain the target image generation model;

[0179] performing semantic feature extraction on the expected content description to obtain an image description vector, and performing semantic feature extraction on the expected image style to obtain an expected image style vector;

[0180] inputting the expected image style vector and the image description vector into the target image generation model to generate an image, and obtaining an image output by the target image generation model as the initial generated image.

[0181] The specific limitations of the data processing apparatus can refer to the limitations of the data processing method described above, which will not be repeated here. Each module in the above data processing apparatus can be implemented by software, hardware and their combination. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0182] In one embodiment, as shown in Figure 7 A computer device is provided, which can be a server, and the computer device includes a processor, a memory, a network interface and a database connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data used and generated by the above data processing method, such as image description information, key image information and initial generated images. The network interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a data processing method.

[0183] In one embodiment, a computer device is provided, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program:

[0184] Obtaining image description information of a desired image from a user, the image description information including desired image style and desired content description;

[0185] Generating an initial generated image based on the desired content description and the desired image style, and performing semantic extraction on the desired content description to obtain key image information corresponding to the desired content description;

[0186] Performing image content recognition on the initial generated image to obtain content recognition data, and performing style recognition on the initial generated image to obtain image recognition style;

[0187] Determining whether the image recognition style is consistent with the desired image style, and determining whether the content recognition data matches the key image information;

[0188] If the image recognition style is consistent with the desired image style, and the content recognition data matches the key image information, the initial generated image is output as a target retrieval image corresponding to the desired image.

[0189] In one embodiment, a computer readable storage medium is provided, and a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement the following steps:

[0190] Obtaining image description information of a desired image from a user, the image description information including a desired image style and a desired content description;

[0191] Generating an initial generated image based on the desired content description and the desired image style, and performing semantic extraction on the desired content description to obtain key image information corresponding to the desired content description;

[0192] Performing image content recognition on the initial generated image to obtain content recognition data, and performing style recognition on the initial generated image to obtain an image recognition style;

[0193] Determining whether the image recognition style is consistent with the desired image style, and determining whether the content recognition data matches the key image information;

[0194] If the image recognition style is consistent with the desired image style, and the content recognition data matches the key image information, then outputting the initial generated image as a target retrieval image corresponding to the desired image.

[0195] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory.

[0196] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0197] The above-described embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A data processing method, characterized by, The method comprises the following steps: obtaining image description information of a desired image, wherein the image description information comprises a desired image style and a desired content description; generating an initial generated image based on the desired content description and the desired image style, and performing semantic extraction on the desired content description to obtain key image information corresponding to the desired content description; performing image content recognition on the initial generated image to obtain content recognition data, and performing style recognition on the initial generated image to obtain an image recognition style; determining whether the image recognition style is consistent with the desired image style, and determining whether the content recognition data matches the key image information; if the image recognition style is consistent with the desired image style and the content recognition data matches the key image information, outputting the initial generated image as a target retrieval image corresponding to the desired image; if the image recognition style is inconsistent with the desired image style or the content recognition data does not match the key image information, generating a new initial generated image according to the image description information until the new initial generated image meets the user description requirement, and then outputting the new initial generated image as the target retrieval image corresponding to the desired image; extracting a key field in the image description information, matching the key field with a label of an existing image in a database to obtain a matched image, and returning the target retrieval image and the matched image as image retrieval results to the user.

2. The data processing method of claim 1, wherein, The key image information comprises a plurality of desired elements of the desired image, and the determination of whether the content recognition data matches the key image information comprises: performing constituent element recognition on the content recognition data to obtain a plurality of constituent elements of the initial generated image; one-to-one matching of the plurality of desired elements with the plurality of constituent elements to obtain a matching result; if the element matching result is complete matching, it is determined that the content recognition data matches the key image information; if the element matching result is not complete matching, it is determined that the content recognition data does not match the key image information.

3. The data processing method of claim 2, wherein, The key image information further comprises a position and a color of each desired element, and the one-to-one matching of the plurality of desired elements with the plurality of constituent elements to obtain a matching result comprises: determining whether the plurality of constituent elements and the plurality of desired elements can be matched one by one; if the plurality of constituent elements and the plurality of desired elements can be matched one by one, determining the position and the color of each constituent element based on the content recognition data; determining whether the position and the color of each constituent element match the position and the color of the corresponding desired element; if the position and the color of each constituent element match the position and the color of the corresponding desired element, it is determined that the matching result is complete matching; if the position or the color of any constituent element does not match the position or the color of the corresponding desired element, it is determined that the matching result is not complete matching.

4. The data processing method of claim 3, wherein, The determining whether the position and color of each of the constituent elements match the position and color of the corresponding expected element comprises: calculating a distance between the position of each of the constituent elements and the position of the corresponding expected element to obtain a position distance of each of the constituent elements; calculating a similarity between the color of each of the constituent elements and the color of the corresponding expected element to obtain a color difference of each of the constituent elements; determining whether the position distance of each of the constituent elements is less than a preset distance, and determining whether the color difference of each of the constituent elements is less than a preset color difference value; if the position distance of each of the constituent elements is less than the preset distance, and the color difference of each of the constituent elements is less than the preset color difference value, it is determined that the position and color of each of the constituent elements match the position and color of the corresponding expected element.

5. The data processing method of claim 2, wherein, The key image information further comprises an expected background description, and after the plurality of expected elements and the plurality of constituent elements are matched one by one to obtain a matching result, the method further comprises: if the matching result is a complete match, performing background recognition on the content recognition data to obtain an image background description of the initial generated image; determining a background similarity between the expected background description and the image background description; if the background similarity is greater than or equal to a preset background similarity, it is determined that the content recognition data matches the key image information; if the background similarity is less than the preset background similarity, it is determined that the content recognition data does not match the key image information.

6. The data processing method of claim 5, wherein, The expected background description comprises an expected background color and an expected background pattern, the image background description comprises a recognized background color and a recognized background pattern, and the determining the background similarity between the expected background description and the image background description comprises: calculating a background color similarity between the expected background color and the recognized background color, and calculating a pattern similarity between the expected background pattern and the recognized background pattern; calculating the background similarity based on the background color similarity and the pattern similarity.

7. The data processing method according to any one of claims 1 to 6, characterized in that, The image generation based on the expected content description and the expected image style to obtain an initial generated image comprises: obtaining a target image generation model obtained by deep learning of a generative adversarial network based on a model sample data set, each historical image sample in the model sample data set corresponding to a sample content description vector and a sample style vector, the target image generation model being obtained by deep learning training of a generative adversarial network based on sample content description vectors and sample style vectors of a plurality of historical image samples; performing semantic feature extraction on the expected content description to obtain an image description vector, and performing semantic feature extraction on the expected image style to obtain an expected image style vector; inputting the expected image style vector and the image description vector into the target image generation model for image generation, and obtaining an image output by the target image generation model as the initial generated image.

8. A data processing apparatus, characterized by, comprises: An acquisition module is configured to acquire image description information of an expected image from a user, the image description information including an expected image style and an expected content description; A generation module is configured to generate an initial generated image based on the expected content description and the expected image style, and to extract semantic information of the expected content description to obtain key image information corresponding to the expected content description; An identification module is configured to identify content of the initial generated image to obtain content identification data, and to identify a style of the initial generated image to obtain an image identification style; A determination module is configured to determine whether the image identification style is consistent with the expected image style, and to determine whether the content identification data matches the key image information; An output module is configured to output the initial generated image as a target search image corresponding to the expected image if the image identification style is consistent with the expected image style and the content identification data matches the key image information; If the image identification style is inconsistent with the expected image style or the content identification data does not match the key image information, a new initial generated image is generated again according to the image description information until the new initial generated image meets the user description requirement, and the new initial generated image is output as the target search image corresponding to the expected image; Key fields in the image description information are extracted, the key fields are matched with labels of existing images in a database to obtain matched images, and the target search image and the matched images are returned to the user as image search results.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the steps of the data processing method according to any one of claims 1 to 7.

Images