Data acquisition method and device, equipment and storage medium

Abstract

The invention provides a data collection method, device and equipment and a storage medium, relates to the technical field of data processing, in particular to the fields of data collection, large models and the like, and can be used for application scenes such as generative search, document intelligent editing, intelligent assistants, virtual assistants, intelligent e-commerce and the like. According to the specific implementation scheme, initial data in an initial data set are obtained in multiple modes, wherein the initial data comprise sample images and text descriptions corresponding to the sample images; and performing comprehensive evaluation on the initial data in the initial data set by adopting a preset evaluation index to perform data cleaning on the initial data set to obtain a target data set. According to the embodiment of the invention, the initial data is obtained in multiple modes, the diversity of the data can be guaranteed, the data is cleaned through the preset indexes, the high-quality training data can be produced, and a better effect can be generated by the text graph model.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to the fields of data collection and large models, and can be used in application scenarios such as generative search, intelligent document editing, intelligent assistants, virtual assistants, and intelligent e-commerce. Background Art

[0002] AI (Artificial Intelligence) painting, as one of the important application fields of artificial intelligence, has made significant breakthroughs in recent years.

[0003] In related technologies, images of various styles can be generated based on the user's natural language description, which not only provides users with powerful auxiliary tools but also brings new possibilities to the field of digital creativity. Summary of the invention

[0004] The present disclosure provides a data collection method, device, equipment and storage medium.

[0005] According to one aspect of the present disclosure, there is provided a data collection method, comprising:

[0006] Using multiple methods to obtain initial data in the initial data set, the initial data includes sample images and text descriptions corresponding to the sample images;

[0007] The preset evaluation indicators are used to comprehensively evaluate the initial data in the initial data set to clean the initial data set and obtain the target data set.

[0008] According to another aspect of the present disclosure, there is provided a data acquisition device, comprising:

[0009] An acquisition module, used to acquire initial data in an initial data set in a variety of ways, wherein the initial data includes a sample image and a text description corresponding to the sample image;

[0010] The screening module is used to use preset evaluation indicators to comprehensively evaluate the initial data in the initial data set, so as to clean the initial data set and obtain the target data set.

[0011] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0012] at least one processor; and

[0013] a memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0015] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0016] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0017] In the disclosed embodiment, initial data is obtained in a variety of ways to ensure data diversity, and data is cleaned by preset indicators to produce high-quality training data, so that the text graph model can generate better results.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0020] Figure 1 is a flow chart of a data collection method according to an embodiment of the present disclosure;

[0021] Figure 2 is a schematic diagram of a process of obtaining a target data set according to an embodiment of the present disclosure;

[0022] Figure 3 is a schematic diagram of a process for improving text descriptions corresponding to sample images in a target data set according to an embodiment of the present disclosure;

[0023] Figure 4 is a schematic diagram of a process for verifying and optimizing a target data set according to an embodiment of the present disclosure;

[0024] Figure 5 is a complete large-scale high-quality data acquisition process diagram according to an embodiment of the present disclosure;

[0025] Figure 6 is a structural schematic diagram of a data acquisition device according to an embodiment of the present disclosure;

[0026] Figure 7 It is a block diagram of an electronic device used to implement the data collection method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0028] The terms "first", "second", etc. in this disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or precedence. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, including a series of steps or units. The method, system, product or device is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] With the development of AI painting, users are no longer limited to existing gallery resources when searching for images or creating works. Instead, they can create new content and image styles by generating images based on trained AI painting applications according to their own needs and creativity.

[0030] In some AI painting applications, massive amounts of data are directly captured from the Internet, simply cleaned, and then fed to the model as training corpus so that the model can generate images based on the user's description. However, the quality of data obtained in this way is difficult to balance, which may ultimately lead to the need to improve the image effects generated by the model.

[0031] In view of this, an embodiment of the present disclosure provides a data collection method, which obtains data through multiple acquisition methods and screens the data from multiple dimensions to obtain a high-quality data set for training a model.

[0032] like Figure 1 FIG. 1 is a flow chart of a data collection method proposed in an embodiment of the present disclosure, including:

[0033] S101, using a variety of methods to obtain initial data in an initial data set, where the initial data includes sample images and text descriptions corresponding to the sample images.

[0034] The initial data set including text description and corresponding sample image data is collected as initial data in various ways. The text description may include at least one of the title, label, descriptive sentence, etc. of the sample image. When the text description is in English and a model for generating images based on Chinese description needs to be trained, the acquired English can be converted into the corresponding Chinese description through machine translation.

[0035] S102, using preset evaluation indicators to comprehensively evaluate the initial data in the initial data set, so as to clean the initial data set and obtain a target data set.

[0036] By using preset evaluation indicators to comprehensively evaluate the initial data set, and based on the comprehensive evaluation results, by cleaning the initial data set, invalid data can be removed, erroneous data can be corrected, and data quality can be improved, thereby improving data consistency and accuracy and obtaining a high-quality target data set.

[0037] In the disclosed embodiments, initial data is obtained from the initial data set in a variety of ways, which can enrich the source and diversity of the data. The sample images and the text descriptions corresponding to the sample images can ensure the comprehensiveness of the initial data content. The obtained initial data is comprehensively evaluated using preset evaluation indicators, and low-quality, irrelevant or inaccurate data can be removed by flexibly setting specific standards, thereby improving the overall quality of the data. This makes the subsequent model training of the text map based on these image data sets more reliable and effective.

[0038] In the embodiments of the present disclosure, the initial data in the initial data set may be obtained in a variety of ways, including:

[0039] Method 1: Get initial data from open source datasets.

[0040] Open source datasets refer to data sets that are publicly available and have no copyright restrictions. These datasets are usually provided by individuals, organizations or research institutions to promote data sharing and scientific research. Open source datasets contain various types of data, such as images, audio, video, text, etc. Obtaining initial data from open source datasets means obtaining various image data from open source datasets. For example, the initial data can be obtained from the video frames corresponding to the audio, with the text converted from the audio as the text description and the video corresponding to the audio as the sample image. This is especially suitable for some science education or explanation video programs, such as popular science videos introducing animal habits, popular science videos introducing plant characteristics, videos that echo the story content, etc. During implementation, you can obtain the required data from the open source dataset according to your own needs.

[0041] Method 2: extract keywords from the request statement of the target object, and search for sample images corresponding to the keywords from the network based on the keywords to construct initial data;

[0042] In the case that no sample image corresponding to the keyword is searched, the keyword is rewritten based on the text rewriting model to obtain a rewritten sentence; and a sample image corresponding to the rewritten sentence is generated based on the rewritten sentence to construct initial data.

[0043] During implementation, the description of the target object for the image it wants to generate is used as a request statement, and the request statement is segmented, and key entity words are extracted from the segmented keywords as keywords. For example, the entity words that can be extracted include some names, place names, dish names, idioms, and vocabulary containing certain IP (Intellectual Property) knowledge. Based on these entity words, large-scale image capture is carried out from major copyright-free image websites on the Internet through web crawling to obtain image data of corresponding concepts. If the images on the website are copyrighted, you can apply for authorization and then obtain the corresponding image data.

[0044] In some embodiments, it may not be possible to find relevant sample images through entity word search in the network. At this time, the data can be supplemented by generating a large text rewriting model and some currently mature image generation models. Among them, the large text rewriting model can accept the original entity word as input and generate at least one of a series of possible synonyms, near synonyms and rewritten sentences as the aforementioned rewritten sentences. The rewritten sentence can be a paraphrase of the original entity word to facilitate better explanation. These rewritten sentences can be used again for network search to find relevant sample images. If enough sample images still cannot be found through network search, the aforementioned image generation model can be considered to understand the rewritten sentence to generate the corresponding image based on the rewritten sentence. Thus, during implementation, the large text rewriting model and the aforementioned image generation model can be jointly trained for fine-tuning so that the rewritten sentence generated by the large text rewriting model can better facilitate the image generation model to understand and reason, so as to generate reasonable high-quality images.

[0045] In the disclosed embodiments, obtaining data from open source datasets can effectively expand the richness and diversity of datasets. Extracting keywords from the request statements of the target object and searching for corresponding sample images from the network based on the keywords can ensure the pertinence and practicality of the data. In the case where sample images corresponding to the keywords are not found, rewriting the keywords through a large text rewriting model and generating sample images based on the rewritten statements can effectively solve the problem of insufficient data and realize targeted data generation. At the same time, it can also introduce new perspectives and creativity to increase the diversity of data.

[0046] After the data is screened out, in order to filter out low-quality images, the data needs to be cleaned according to preset evaluation indicators, wherein the preset evaluation indicators may include at least one of the following:

[0047] (1) Image properties of sample images.

[0048] The image attributes of the sample image refer to some basic features of the image itself, such as the aspect ratio and image format of the image itself.

[0049] (2) User preference attributes of sample images.

[0050] The user attribute preference of the sample image is the target object's positive feedback attribute to the sample image and / or the target object's negative feedback attribute to the sample image, that is, it reflects the target object's subjective preference and selection tendency for the sample image. For example, the target object's likes and dislikes of certain sample images. Through the target object's positive feedback attribute and / or negative feedback attribute to the sample image, the target data set that currently meets the public's preferences can be screened out. In order to make the target data set meet the general aesthetics and meet the current aesthetic needs.

[0051] In the disclosed embodiment, the positive feedback attribute of the target object to the sample image is expressed by at least one of the operation information such as forwarding operation information, liking operation information, sharing operation information, collecting operation information, downloading operation information, etc., which indicates the target object's liking for the sample image. Through these data, it is possible to more accurately identify which images are more popular with users, more attractive or more valuable.

[0052] It should be noted that the positive feedback attributes of the target object to the sample image are not limited to the attributes listed above, and can be flexibly determined according to actual needs, as long as they can express the user's preference for the sample image.

[0053] (3) Image quality of sample images.

[0054] The image quality of the sample image includes image clarity, aesthetics, resolution, etc. Among them, the aesthetics can be scored by using a pre-trained aesthetics evaluation network model. The higher the score, the higher the aesthetics of the sample image. In addition, the aesthetics scores of different sample images can also be obtained through questionnaires.

[0055] Among them, the higher the clarity and resolution, the better the quality of the generated image. Therefore, using images with high clarity and resolution to train the Vincent Image model will improve the quality of the images generated by the model.

[0056] (3) Painting style

[0057] The painting style refers to whether it is the painting style required for training, such as realistic, abstract, figurative, etc. The classification of painting styles is conducive to screening the data of the required painting style for model training.

[0058] (4) The target category to which the sample image content belongs.

[0059] The target category to which the sample image content belongs refers to the category of objects or scenes mainly presented in the image. For example, the image can be divided into categories such as people, animals, and plants.

[0060] The target category information is helpful in screening the required category of data for model training.

[0061] (5) The degree of text-image matching between the text description in the initial data and the sample image.

[0062] The degree of text-image matching between the text description in the initial data and the sample image is the degree of fit between the content presented in the image and the text description of the image. In implementation, the sample image and the text description can be mapped to the same vector space to obtain the image features of the sample image and the text features of the text description. The degree of text-image matching between the text description and the sample image can be expressed by determining the similarity between the image features and the text features.

[0063] It is understandable that the higher the image-text match, the more the image content fits the text description. Such data will help improve the generation quality of the large text-image model.

[0064] In the disclosed embodiments, by presetting evaluation indicators, the required data set can be screened from a single perspective. The data set can also be cleaned from multiple perspectives to comprehensively and systematically screen out high-quality images that perform well in all dimensions to meet different needs and application scenarios. The image attributes of the sample images can ensure that the generated images are of various sizes. The user preferences of the sample images can ensure that the generated images meet the user's preferences. The image quality of the sample images can ensure the quality of the generated images. The style of the sample images and the target category to which the sample image content belongs can ensure data balance.

[0065] In some embodiments, there may be some high-quality images that meet preset conditions but have other problems, such as black borders added to maintain the aspect ratio of the original image, and original information watermarks retained in authorized images. For such images, feature coverage of the image can be achieved by constructing a quality operator to ensure the integrity of the image. For example, the operator for evaluating the image quality of the sample image may include a black border detection operator, and the black border detection operator may be used to detect the black border in the sample image; if there is a black border in the sample image, the black border in the sample image is repaired.

[0066] The black edge detection operator is used to detect the black edges in the sample image, that is, to determine whether there is a black border on the edge of the sample image through a specific algorithm or calculation method. When the image is detected to have a black border, the relevant image processing technology will be used to repair the black border part, which may include cropping the black border, or adjusting the color, brightness, etc. of the black border area to make it better integrated and connected with other parts of the image, thereby improving the overall quality and visual effect of the image.

[0067] In the disclosed embodiment, by detecting and repairing black edges, the influence of black edges on the overall aesthetics of the image and the display of the content can be avoided. Repairing black edges can make the edges of the image more natural and regular, and can exclude images whose quality is reduced by black edges, thereby improving the overall quality and visual consistency of the image, and ensuring that the selected images reach a high level in appearance and integrity.

[0068] Correspondingly, the operator for evaluating the image quality of the sample image may also include a watermark detection operator, and the watermark detection operator may be used to detect the watermark in the sample image; when a watermark exists in the sample image, the watermark in the sample image is removed.

[0069] The watermark detection operator can analyze the characteristics of the image to find possible watermarks. In the case of watermarks in the sample image, appropriate image processing techniques are used to remove these watermarks. Through methods such as pixel modification, color adjustment or pattern overlay, the watermark is visually eliminated or becomes invisible, thereby restoring the original clarity and beauty of the image.

[0070] In the disclosed embodiment, by detecting and removing watermarks, images can be presented in a more original state, and the clarity and purity of the images can be improved, thereby ensuring that the screened images have higher quality.

[0071] After the image is covered with features, the above preset indicators are used to comprehensively evaluate the initial data in the initial data set to clean the initial data set and obtain the target data set, such as Figure 2 As shown, including:

[0072] S201, when the preset evaluation index includes multiple indexes, scoring the initial data in the initial data set based on the multiple indexes to obtain a comprehensive score of each initial data.

[0073] During implementation, for each initial data, various indicators of the initial data are mapped to the value range of each indicator to obtain the evaluation value of each indicator; the evaluation value of each indicator is weighted summed to determine the comprehensive score of the initial data.

[0074] Among them, the comprehensive score has a positive correlation with the first target attribute in the image quality for improving the quality of the generated image; the comprehensive score has a positive correlation with the second target attribute in the user preference attribute representing the user's preference.

[0075] By combining historical data or industry standards, the value range or standard interval of each indicator is clarified, and it is used to represent the performance level of each indicator in the overall evaluation. After obtaining the evaluation value of each indicator, each indicator is weighted and summed according to the assigned weight to obtain the comprehensive score of the initial data. Comprehensively scoring the initial data based on multiple indicators can comprehensively reflect the performance level of the initial data in various aspects. By comparing the comprehensive scores of different initial data, they can be sorted and compared, thereby screening out higher quality initial data.

[0076] During implementation, for certain subclassified data sets, corresponding data ranges can be set for the corresponding subclassifications. For example, the value interval of the weight or subclassification can be set to a larger specified interval, and the specified area can be significantly different from other subclassifications. The value intervals of other preset indicators outside the subclassification are smaller value intervals so that the weighted sum of other evaluation indicators will not exceed the specified value, so that the required subclassified data sets can be screened out through the larger value interval of the subclassification. For example, the value interval of the subclassification of a certain painting style can be set to (100, 200), the value interval of the face can be set to (300, 400), and the value range of other evaluation indicators is (0-1) so that the weighted sum of other evaluation indicators will not exceed the specified value, such as 10. In this way, for a specific painting style, assuming that its value interval is (100, 120), high-scoring data with a comprehensive score in this value interval can be selected to obtain a high-quality data set that meets the requirements of the painting style.

[0077] In addition, different preset indicators can be restricted to occupy different binary bits through binary bits to express the value range of different preset indicators. For example, the total number of binary bits can be set according to the total value length of the preset indicators. Among them, different indicators occupy different binary bits. Then, the binary bits of each indicator are used to screen different subcategories, and then for the initial data of the same subcategories, the data quality of the subcategories is expressed by weighted sum (i.e., comprehensive score).

[0078] S202: Based on the comprehensive score, the initial data set is cleaned to obtain a target data set.

[0079] That is, after obtaining the comprehensive score of each data, the initial data is cleaned according to these scores. For example, if the comprehensive score of a certain data is lower than the preset score value, the data is discarded, and so on. The remaining initial data that meets the preset score value is used as the target data set.

[0080] In the disclosed embodiment, by scoring through multiple indicators, the quality of the image can be comprehensively evaluated from multiple angles to obtain a comprehensive score, so that the quality of the image can be judged more objectively and accurately. Data cleaning of the initial data based on the comprehensive score can effectively remove images that perform poorly in all aspects and do not meet the high quality requirements, thereby screening out higher quality images as the target data set.

[0081] For the target data set after data cleaning, in order to optimize the data collection and use process, the distribution of data of different levels can be further analyzed, and the initial data in the target data set can be graded based on the comprehensive score of the initial data in the target data set. Different levels represent different data quality.

[0082] According to the comprehensive score, the data is divided into different levels. The number of levels and the division criteria can be determined according to actual needs. For example, the data can be divided into three levels: high, medium, and low, or N levels in more detail. When dividing, the rationality of the score interval can be considered to ensure that the data of each level has obvious characteristics and value differences.

[0083] In the disclosed embodiment, by classifying the initial data in the target data set, different data qualities can be located more accurately, facilitating subsequent use and management; high-quality images that meet specific requirements can be quickly screened out, thereby improving work efficiency.

[0084] In the embodiment of the present disclosure, in the case where the text description corresponding to the sample image in the acquired target data set is incomplete, the text description information can be further improved, such as Figure 3 As shown, including the following:

[0085] S301, screening out sample images to be processed from an initial data set or a target data set.

[0086] Among them, if the sample images to be processed are screened out from the initial data set, the data quality of the initial data set can be improved, so as to facilitate the subsequent expansion of the data volume of the target data set.

[0087] Of course, according to actual needs, the initial data set can be cleaned first to obtain the target data set. In this case, the target data set obtained has higher data quality, and then the following method of the embodiment of the present disclosure is used to accurately describe the sample image to further improve the data quality.

[0088] S302, performing a labeling operation on the sample image to be processed to obtain an image label of the sample image to be processed.

[0089] During implementation, the multi-label model can be used to perform label recognition on sample images. For example, after labeling a sample image to be processed, the labels obtained are "puppy, grass, child, blue sky, running".

[0090] S303: performing a classification operation on the sample image to be processed to obtain category information of the sample image to be processed.

[0091] By constructing an image classification operator, the sample images to be processed are labeled with corresponding categories, such as people, animals, scenery, etc.

[0092] S304, generating image description information of the sample image to be processed in a generative manner.

[0093] For example, by constructing an image description operator, a detailed natural language description of the sample image to be processed can be made.

[0094] It should be noted that the execution timing of step S302, step S303 and step S304 is not limited. The image label, category information and image description information are used to update the text description of the sample image to be processed.

[0095] In the disclosed embodiments, the text description of the sample image can be further improved by labeling, classifying, generating text description information, etc., thereby ensuring the correlation between the sample image and the corresponding text description in the target data set, further improving the data quality, and generally improving the training effect of the large text image model.

[0096] In the embodiment of the present disclosure, initial data is obtained from open source websites by searching or generating keywords, and the initial data is cleaned by using preset evaluation indicators to obtain a high-quality target data set. The obtained target data set needs to be further verified to see whether the quality of the target data set meets the requirements for model training. The specific verification method is as follows: Figure 4 As shown, including the following:

[0097] S401, after training the text-image model based on the target data set, randomly extract multiple text samples used to generate images of the target object.

[0098] In order to improve data quality and facilitate subsequent targeted training of the text graph model, the text samples here can be real texts of the target object's actual operations.

[0099] S402, determining the image-text matching degree between a plurality of text samples and the to-be-evaluated images respectively generated by the text-image large model based on the plurality of text samples.

[0100] S403, based on the image-text matching degrees respectively corresponding to the multiple text samples, the generation quality of each image category in the multiple image categories covered by the multiple text samples by the text-to-image large model is statistically calculated.

[0101] S404, screening image categories with generation quality lower than expected quality as categories to be expanded.

[0102] S405, generating a data set of the category to be expanded, and using the data set of the category to be expanded to optimize the large model of the text graph.

[0103] The selected target data set is used to train a large text-image model. By randomly extracting text descriptions corresponding to sample images in the target data set, the extracted multiple text samples are input into the trained large text-image model to generate corresponding images to be evaluated.

[0104] For the generated images to be evaluated, appropriate evaluation methods, such as manual evaluation, automated evaluation tools, etc., are used to determine the text-image matching degree between each text sample and the generated image to be evaluated. Based on the text-image matching degrees corresponding to multiple text samples, the generation quality of the text-image model in each image category is statistically analyzed. For example, the text-image matching degrees in each image category can be averaged or weighted to obtain the overall generation quality score of the category.

[0105] According to the statistical results, the image categories with generation quality lower than the expected quality are selected as the categories to be expanded. For the selected categories to be expanded, a new data set is generated.

[0106] In the disclosed embodiment, the text-image model is trained by the target data set, and the text samples in the target data set are randomly sampled to detect the text-image matching between the text samples and the generated images, so that which image categories in the target data set perform poorly can be identified. The data set can be further supplemented and optimized for this category, thereby improving the image quality in the target data set.

[0107] During implementation, for each category to be expanded for each text sample in the data set of the category to be expanded, the corresponding sample image can be generated based on the following method:

[0108] Step A1: extract category information of the category to be expanded from the text sample.

[0109] Extract key information of the category, such as category name, subcategory, keywords, etc., from the text samples in the dataset of the category to be expanded.

[0110] Step A2: Generate multiple descriptive words belonging to the category to be expanded based on the large language model for the category information.

[0111] For the extracted category information of the category to be expanded, the category information is processed using a large language model, for example, a natural language description is generated based on multiple keywords, and the generated natural language description can accurately reflect the input content.

[0112] Step A3: Based on each of the multiple description words, generate a sample image corresponding to each description.

[0113] Based on the generated multiple description words, corresponding sample images are generated respectively. In implementation, the corresponding image set can be retrieved from the open source website through each generated description word. Alternatively, each description word can be converted into a corresponding image by using the aforementioned image generation model.

[0114] In the disclosed embodiment, by extracting category information from text samples and generating description words using a large language model, it is possible to ensure that the generated description words are highly relevant to the categories to be expanded, so that when sample images are generated based on these description words, images closely related to the categories can be obtained. This helps to improve the quality and relevance of the entire data set.

[0115] In summary, the data acquisition method provided in the embodiments of the present disclosure can obtain initial data in a variety of ways, and select high-quality initial data from a variety of dimensions as the target data set. Figure 5 As shown, including:

[0116] S501, initial data is obtained in a variety of ways, that is, large-scale graphic and text data is obtained through various means, such as site crawling, directional query (keyword) crawling, directional generation and other solutions.

[0117] S502, the initial data is machine cleaned based on preset indicators to obtain a candidate data set, that is, by constructing some image quality operators, such as black edge, clarity, beauty, watermark and other operators, the quality of the image is identified, thereby filtering out low-quality operators.

[0118] S503, accurately label the candidate data, that is, use some image classification operators, image label operators, and image caption (an algorithm for generating image descriptions) operators to label the images with detailed categories, labels, and descriptions.

[0119] S504, based on the preset aesthetic standards, the target data set is screened out, that is, the high-quality data screened by the machine is sent to manual annotation to screen out aesthetic data that meets the aesthetic standards to further improve the model effect.

[0120] During implementation, the image-text pair accurately marked in S503 can be output and displayed to the marked object, and multiple dimensions of evaluation can be output simultaneously, so that the marked object can score these multiple dimensions according to unified aesthetic knowledge. Then, the aesthetic score of the image-text pair can be obtained according to the scoring results of multiple marked objects on the same image-text pair. Based on this method, better quality data can be screened out to facilitate more accurate data classification.

[0121] S505, training the model with the target data set, and optimizing the target data set based on the generated results. That is, using the screened target data set to train the text graph model, and then using commonly used data for reasoning, according to the model reasoning results, statistics are collected on the data categories with poor reasoning effects in the target data set, and then the corresponding data is supplemented in a targeted manner.

[0122] In summary, the disclosed embodiments can produce high-quality training data on a large scale, so that the Wenshengtu model can generate better results. The Wenshengtu model with the technology of this embodiment as the core can increase the number of clicks and downloads of the product, increase the number of active creations, and obtain more users. Therefore, the DAU (Daily Active Users) and retention data of the application can be improved.

[0123] Based on the same technical concept, the embodiment of the present disclosure also provides a data acquisition device 600, such as Figure 6 As shown, including:

[0124] An acquisition module 601 is used to acquire initial data in an initial data set in a variety of ways, where the initial data includes a sample image and a text description corresponding to the sample image;

[0125] The screening module 602 is used to use preset evaluation indicators to comprehensively evaluate the initial data in the initial data set, so as to clean the initial data set and obtain the target data set.

[0126] In some embodiments, the preset evaluation index includes at least one of the following:

[0127] Image attributes of sample images, user preference attributes of sample images, image quality and style of sample images, target category to which the sample image content belongs, and the image-text matching degree between the text description in the initial data and the sample images.

[0128] In some embodiments, the operator for evaluating the image quality of the sample image includes a black edge detection operator, and the apparatus further includes a first quality processing module for:

[0129] The black edge detection operator is used to detect the black edges in the sample image;

[0130] If there are black edges in the sample image, the black edges in the sample image are repaired.

[0131] In some embodiments, wherein the operator for evaluating the image quality of the sample image includes a watermark detection operator, the apparatus further includes a second quality processing module for:

[0132] Using watermark detection operator to detect watermark in sample image;

[0133] When a watermark exists in the sample image, the watermark in the sample image is removed.

[0134] In some embodiments, the screening module comprises:

[0135] A scoring unit, used for scoring the initial data in the initial data set based on the multiple indicators to obtain a comprehensive score of each initial data when the preset evaluation indicator includes multiple indicators;

[0136] The cleaning unit is used to clean the initial data set based on the comprehensive score to obtain the target data set.

[0137] In some embodiments, the scoring unit is specifically used to:

[0138] For each initial data, various indicators of the initial data are mapped to the value range of each indicator to obtain the evaluation value of each indicator;

[0139] The evaluation value of each indicator is weighted and summed to determine the comprehensive score of the initial data.

[0140] In some embodiments, it also includes:

[0141] The classification module is used to classify the initial data in the target data set based on the comprehensive scores of the initial data in the target data set.

[0142] In some embodiments, the user preference attributes of the sample image include at least one of the following:

[0143] The positive feedback attribute of the target object to the sample image and / or the negative feedback attribute of the target object to the sample image.

[0144] In some embodiments, the positive feedback attribute includes at least one of the following information for the sample image:

[0145] Forwarding operation information, liking operation information, sharing operation information, collecting operation information, and downloading operation information.

[0146] In some embodiments, an optimization module is further included for:

[0147] After training the text-generated image model based on the target dataset, multiple text samples used to generate images of the target object are randomly extracted;

[0148] Determine the image-text matching degree between the multiple text samples and the images to be evaluated generated by the text-image large model based on the multiple text samples;

[0149] Based on the image-text matching degree corresponding to multiple text samples, the generation quality of each image category in the multiple image categories covered by multiple text samples is statistically calculated.

[0150] Screen out image categories whose generated quality is lower than expected quality as categories to be expanded;

[0151] Generate a data set of the category to be expanded, so as to optimize the Wenshengtu model using the data set of the category to be expanded.

[0152] In some embodiments, the optimization module is specifically used to:

[0153] For each text sample in the data set of the category to be expanded;

[0154] Extract category information of the category to be expanded from the text sample;

[0155] Generate multiple descriptive words belonging to the category to be expanded based on the category information based on the large language model;

[0156] Based on each of the plurality of descriptors, a sample image corresponding to each of the descriptions is generated.

[0157] In some embodiments, an update module is further included, which is used to:

[0158] Filter out sample images to be processed from the initial data set or the target data set;

[0159] Perform labeling operation on the sample image to be processed to obtain the image label of the sample image to be processed; and

[0160] Performing a classification operation on the sample image to be processed to obtain category information of the sample image to be processed; and,

[0161] Generate image description information of the sample image to be processed in a generative manner;

[0162] The image label, category information and image description information are used to update the text description of the sample image to be processed.

[0163] In some embodiments, the acquisition module is specifically used to acquire the initial data in the initial data set based on the following method:

[0164] Method 1: Obtain initial data from open source datasets;

[0165] Method 2: extract keywords from the request statement of the target object, and search for sample images corresponding to the keywords from the network based on the keywords to construct initial data;

[0166] In the case that no sample image corresponding to the keyword is searched, the keyword is rewritten based on the text rewriting model to obtain a rewritten sentence; and a sample image corresponding to the rewritten sentence is generated based on the rewritten sentence to construct initial data.

[0167] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0168] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0169] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0170] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0171] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0172] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0173] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as data acquisition methods. For example, in some embodiments, the data acquisition method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the data acquisition method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the data acquisition method in any other appropriate manner (e.g., by means of firmware).

[0174] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0175] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0176] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0177] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0178] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0179] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0180] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0181] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data collection method, comprising: Acquire initial data in an initial data set in a variety of ways, wherein the initial data includes a sample image and a text description corresponding to the sample image; The initial data in the initial data set is comprehensively evaluated using a preset evaluation index to clean the initial data set to obtain a target data set.

2. The method according to claim 1, wherein: The preset evaluation index includes at least one of the following: Image attributes of sample images, user preference attributes of sample images, image quality and style of sample images, target category to which the sample image content belongs, and the image-text matching degree between the text description in the initial data and the sample images.

3. The method according to claim 2, wherein: The operator for evaluating the image quality of the sample image includes a black edge detection operator, and the method further includes: Using a black edge detection operator to detect black edges in the sample image; When there are black edges in the sample image, the black edges in the sample image are repaired.

4. The method according to claim 2, wherein: The operator for evaluating the image quality of the sample image includes a watermark detection operator, and the method further includes: Using a watermark detection operator to detect a watermark in the sample image; In the case where a watermark exists in the sample image, the watermark in the sample image is removed.

5. The method according to claim 2, wherein: The using of preset evaluation indicators to comprehensively evaluate the initial data in the initial data set to clean the initial data set to obtain a target data set includes: In the case where the preset evaluation index includes multiple indexes, scoring the initial data in the initial data set based on the multiple indexes to obtain a comprehensive score of each initial data; Based on the comprehensive score, the initial data set is cleaned to obtain a target data set.

6. The method according to claim 5, wherein: When the preset evaluation index includes multiple indexes, scoring the initial data in the initial data set based on the multiple indexes to obtain a comprehensive score of each initial data includes: For each initial data, multiple indicators of the initial data are respectively mapped to the value range of each indicator to obtain the evaluation value of each indicator; The evaluation value of each indicator is weighted and summed to determine the comprehensive score of the initial data.

7. The method according to claim 5, further comprising: The initial data in the target data set is graded based on the comprehensive score of the initial data in the target data set.

8. The method according to claim 2, wherein: The user preference attributes of the sample image include at least one of the following: The positive feedback attribute of the target object to the sample image and / or the negative feedback attribute of the target object to the sample image.

9. The method according to claim 8, wherein: The positive feedback attribute includes at least one of the following information for the sample image: Forwarding operation information, liking operation information, sharing operation information, collecting operation information, and downloading operation information.

10. The method according to any one of claims 1 to 9, further comprising: After training the text-based graph model based on the target data set, randomly extracting a plurality of text samples used to generate an image of the target object; Determine the image-text matching degree between the multiple text samples and the images to be evaluated respectively generated by the text-image large model based on the multiple text samples; Based on the image-text matching degrees respectively corresponding to the multiple text samples, the generation quality of each image category in the multiple image categories covered by the multiple text samples by the text-image macro model is statistically calculated; Screen out image categories whose generated quality is lower than expected quality as categories to be expanded; Generate a data set of the category to be expanded, and use the data set of the category to be expanded to optimize the Wensheng graph model.

11. The method according to claim 10, wherein: The step of generating the data set of the category to be expanded includes: For each text sample in the data set of the category to be expanded; Extracting category information of the category to be expanded from the text sample; Generating a plurality of descriptive words belonging to the to-be-expanded category for the category information based on the large language model; Based on each of the plurality of descriptive words, a sample image corresponding to each description is generated.

12. The method according to claim 1, further comprising: Filtering sample images to be processed from the initial data set or the target data set; Performing a labeling operation on the sample image to be processed to obtain an image label of the sample image to be processed; as well as, Performing a classification operation on the sample image to be processed to obtain category information of the sample image to be processed; and, Generate image description information of the sample image to be processed in a generative manner; The image label, the category information and the image description information are used to update the text description of the sample image to be processed.

13. The method according to claim 1, wherein: The initial data in the initial data set is obtained in a variety of ways, including: Method 1, obtaining the initial data from an open source dataset; Method 2, extracting keywords from the request sentence of the target object, and searching the network for sample images corresponding to the keywords based on the keywords to construct the initial data; In the case that the sample image corresponding to the keyword is not searched, the keyword is rewritten based on the text rewriting model to obtain a rewritten sentence; and a sample image corresponding to the rewritten sentence is generated based on the rewritten sentence to construct the initial data.

14. A data acquisition device, comprising: An acquisition module, used to acquire initial data in an initial data set in a variety of ways, wherein the initial data includes a sample image and a text description corresponding to the sample image; The screening module is used to use preset evaluation indicators to comprehensively evaluate the initial data in the initial data set, so as to clean the initial data set and obtain the target data set.

15. The device according to claim 14, wherein: The preset evaluation index includes at least one of the following: Image attributes of sample images, user preference attributes of sample images, image quality and style of sample images, target category to which the sample image content belongs, and the image-text matching degree between the text description in the initial data and the sample images.

16. The device according to claim 15, wherein: The operator for evaluating the image quality of the sample image includes a black edge detection operator, and the device also includes a first quality processing module, which is used to: Using a black edge detection operator to detect black edges in the sample image; When there are black edges in the sample image, the black edges in the sample image are repaired.

17. The device according to claim 15, wherein: The operator for evaluating the image quality of the sample image includes a watermark detection operator, and the device further includes a second quality processing module, which is used to: Using a watermark detection operator to detect a watermark in the sample image; In the case where a watermark exists in the sample image, the watermark in the sample image is removed.

18. The device according to claim 15, wherein: The screening module comprises: A scoring unit, configured to score the initial data in the initial data set based on the multiple indicators to obtain a comprehensive score for each initial data when the preset evaluation indicator includes multiple indicators; A cleaning unit is used to clean the initial data set based on the comprehensive score to obtain a target data set.

19. The device according to claim 18, wherein: The scoring unit is specifically used for: For each initial data, multiple indicators of the initial data are respectively mapped to the value range of each indicator to obtain the evaluation value of each indicator; The evaluation value of each indicator is weighted and summed to determine the comprehensive score of the initial data.

20. The apparatus of claim 18, further comprising: The classification module is used to classify the initial data in the target data set based on the comprehensive score of the initial data in the target data set.

21. The device according to claim 15, wherein: The user preference attributes of the sample image include at least one of the following: The positive feedback attribute of the target object to the sample image and / or the negative feedback attribute of the target object to the sample image.

22. The device according to claim 21, wherein The positive feedback attribute includes at least one of the following information for the sample image: Forwarding operation information, liking operation information, sharing operation information, collecting operation information, and downloading operation information.

23. The apparatus according to any one of claims 14 to 22, further comprising an optimization module, configured to: After training the text-based graph model based on the target data set, randomly extracting a plurality of text samples used to generate an image of the target object; Determine the image-text matching degree between the multiple text samples and the images to be evaluated respectively generated by the text-image large model based on the multiple text samples; Based on the image-text matching degrees respectively corresponding to the multiple text samples, the generation quality of each image category in the multiple image categories covered by the multiple text samples by the text-image macro model is statistically calculated; Screen out image categories whose generated quality is lower than expected quality as categories to be expanded; Generate a data set of the category to be expanded, and use the data set of the category to be expanded to optimize the Wensheng graph model.

24. The device according to claim 23, wherein: The optimization module is specifically used for: For each text sample in the data set of the category to be expanded; Extracting category information of the category to be expanded from the text sample; Generating a plurality of descriptive words belonging to the to-be-expanded category for the category information based on the large language model; Based on each of the plurality of descriptive words, a sample image corresponding to each description is generated.

25. The apparatus according to claim 14, further comprising an updating module, configured to: Filtering sample images to be processed from the initial data set or the target data set; Performing a labeling operation on the sample image to be processed to obtain an image label of the sample image to be processed; and Performing a classification operation on the sample image to be processed to obtain category information of the sample image to be processed; and, Generate image description information of the sample image to be processed in a generative manner; in, The image label, the category information, and the image description information are used to update the text description of the sample image to be processed.

26. The device according to claim 14, wherein: The acquisition module is specifically used to acquire the initial data in the initial data set based on the following method: Method 1, obtaining the initial data from an open source dataset; Method 2, extracting keywords from the request sentence of the target object, and searching the network for sample images corresponding to the keywords based on the keywords to construct the initial data; In the case that the sample image corresponding to the keyword is not searched, the keyword is rewritten based on the text rewriting model to obtain a rewritten sentence; and a sample image corresponding to the rewritten sentence is generated based on the rewritten sentence to construct the initial data.

27. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 13.

28. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-13.

29. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 13.

Images