A method for generating images and texts for crowd-testing task publishing
By using rules and template-based methods on the crowd test platform combined with deep learning technology to generate task text descriptions, and generating style transfer images through generative adversarial networks, the readability and interaction of task descriptions and image generation in the prior art are solved. sexual problems achieve a more intelligent and interactive task release process.
Patent Information
- Application Number
- CN202011373051.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-30
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2040-11-30
AI Technical Summary
In the crowd testing platform, it is difficult for the existing technology to generate readable and logical task descriptions and task pictures, which affects the visualization and interactivity of task release. The existing methods require manual feature engineering and rule intervention, which is inefficient.
Using rules and template-based methods combined with deep learning technology, using structured task data and expert feature corpus to generate task text descriptions, and generating style transfer images through generative adversarial networks and computer vision technologies, including data acquisition, preprocessing, and generation tasks. Text and word cloud graphs, and unsupervised training style transfer model.
It realizes the generation of smooth and interpretable task content descriptions and visual task pictures on the crowd test platform, improves the visualization and interactivity of task release, reduces management pressure, promotes user interaction, and improves task browsing efficiency.
Smart Images

Figure CN112634405B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image and text generation, and is mainly applied to task publishing technology in crowd testing, and specifically to generating fluent task text according to structured data and generating task images according to relevant task text descriptions. Background Art
[0002] With the popularity of smart devices, crowd testing has become a new transformative platform that can attract mobile users to perform spatiotemporal tasks by physically traveling to designated locations. Many researchers have studied various crowd testing technologies for performance optimization, among which how to generate tasks more intelligently and interactively is a current technical challenge. With the rapid development of big data capture, management and processing related technologies and computer vision and natural language processing, related image text generation technologies are becoming increasingly mature. Using the above technologies to generate distinctive task images and readable and logical task descriptions during task release is both a challenge and a possibility.
[0003] With the popularity of intelligent crowd-testing platforms, people's daily work, employment, research, manufacturing and marketing methods are quietly changing, which has a great impact. Among them, task allocation or staff selection is a major issue, which may affect the completion quality of crowd-testing tasks. More visual interaction with people participating in crowd-testing and intelligent task release are both novel and meaningful.
[0004] In the fields of computer vision and natural language, deep learning has achieved many excellent results. Since the application of deep learning in computer vision, many excellent collisions have occurred in the field of vision. There have been great breakthroughs in traditional visual directions such as object recognition, object detection, and video analysis. Many excellent works have been produced in image generation tasks with generative adversarial networks as the main technical route. The present invention is aimed at the task release scenario under crowd testing. During the task release process, the relevant image generation technology is used, combined with crowd testing related technologies, and the model is trained to obtain task pictures with certain characteristics related to the task.
[0005] In recent years, data-to-text generation technology has also achieved some technical results. It uses structured data such as a table as input and generates appropriate and fluent text as output to describe the data. The relevant mainstream methods include rule-based and template-based methods and neural network-based methods. Among them, rule-based and template-based methods are still the mainstream application methods due to their controllability and logic. However, this type of method also requires artificial feature engineering extraction and rule intervention. In the task release module of the present invention in the crowd testing scenario, according to the structured task word description, it obtains knowledge through cooperation with experts or from the corpus generated by experts to generate a logical and fluent task description. Summary of the invention
[0006] Purpose of the invention: To generate a fluent and explainable task content description based on structured related words for relevant task releases in crowd testing; to establish a dedicated task graph for relevant task descriptions, to complete the image and text generation work for crowd testing tasks, and to make crowd testing tasks more visual and interactive. The present invention provides a method for generating images and texts for crowd testing task releases, which uses structured task data and relevant expert feature corpus and natural language processing related technologies to generate a reasonable task text description; uses keyword extraction and word cloud visualization and other technologies to generate a word cloud graph of relevant tasks; uses key technologies such as generative adversarial networks and computer vision to transfer relevant styles and generate a style task graph.
[0007] Technical solution: To achieve the above purpose, the technical solution adopted by the present invention is:
[0008] Compared with the prior art, the present invention has the following beneficial effects:
[0009] A method for generating images and texts for crowd-testing task releases. In the data collection stage, sufficient relevant data from real crowd-testing scenario task releases is obtained, including structured task key fields, detailed descriptions of task releases, and relevant content images or logo image data provided by the task publisher. In the data preprocessing stage, data is sorted and cleaned for different tasks. In the task text description generation stage, a high-quality complete task text description is generated using a rule- and template-based method based on the preprocessed structured task-related words. In the word cloud image generation stage, a word cloud image related to the task is generated based on the task description and related background images such as logo images provided by the relevant task party. In the related style image generation stage, a statistically obtained related style data set is used to perform unsupervised training of a style transfer model using the MUNIT network model. The specific steps include the following:
[0010] Step 1, data collection: obtain relevant data in the task release of the crowd testing scenario, including structured task key fields, detailed description of the task release, and relevant content pictures or logo pictures provided by the task publisher. They are used for structured task release text generation and task release picture generation, respectively. Task release picture generation includes word cloud pictures and relevant task style transfer pictures.
[0011] Step 2: Data preprocessing: organize and clean the data for different tasks.
[0012] Step 3: Generate task text description: Generate high-quality and complete task text description using rule-based and template-based methods based on the preprocessed structured task key fields:
[0013] Step 3a), count the key fields of structured tasks, including task name, description, task type, task reward, deadline, and number of people required.
[0014] Step 3b) uses expert intervention to set relevant rules and templates. The main content of the template includes relevant emotional words, some humorous words, and connecting word sequences. The focus of the template is determined according to different tasks.
[0015] Step 3c), based on the input structured task key fields, use relevant expert rules and template strategies to generate several text messages. The focus of this link is to select appropriate templates according to different tasks to ensure that the content is introduced appropriately while increasing the fluency of the entire text description. By extracting relevant task keywords, determine the overall style of the entire task description, whether there is a focus, and improve the text output of the entire task release.
[0016] Step 4, generate word cloud: Generate task-related word cloud according to the task description and related content pictures or logo pictures provided by the task party. Add dynamic modification method on the basis of generating word cloud, visually adjust color distribution, adjust the number and importance of related words, merge unnecessary related word repetitions, and make the whole visual word cloud more concise and reasonable:
[0017] Step 4a), perform relevant Chinese word segmentation on the task description provided by the relevant task party, set stop words, and perform word frequency statistics.
[0018] Step 4b), process the relevant task pictures or logo pictures provided by the task party, and select a suitable area as the filling background template of the word cloud.
[0019] Step 4c), using the word cloud visualization method, set the relevant font, color, and number of displayed words, use the processed image obtained in step 4b) as the word cloud background, and use the relevant task description word segmentation results in step 4a) as the word cloud content.
[0020] Step 4d), dynamically adjust the generation effect of the word cloud map, visually adjust the color distribution, adjust the number and importance of related words, merge unnecessary repetitions of related words, and make the entire visual word cloud map more concise and reasonable.
[0021] Step 5: Generate relevant style images, use the statistically obtained relevant style dataset, and use the MUNIT network model in the generative adversarial network to perform unsupervised training of the style transfer model.
[0022] Preferred: Data preprocessing method in step 2:
[0023] In step 2a), structured keywords required to generate a fluent task text description are processed.
[0024] Step 2b), conduct relevant statistics on the text descriptions required for word cloud image generation, clean them, and remove unreasonable parts.
[0025] Step 2c), performing correlation processing on the source domain image and the target domain image required for style transfer map generation, and the target domain image is obtained from the oil painting style image, emoji image and anime dimension image dataset or related crawlers.
[0026] Step 2d), save the processed data or data set and use it in subsequent tasks.
[0027] Preferably, the data format of the structured task keywords in step 3 is {'entity, 'description', 'type', 'salary', 'deadline', 'number of staff required', 'number of tasks'}. The structured task keywords are used as input to the rule template to generate a suitable task description.
[0028] Preferably, the step 5 of generating relevant style images can be abstractly summarized as using GAN to achieve multimodal image translation under unpaired data set conditions. The MUNIT network structure model is adopted, and the MUNIT network structure model architecture is auto-encoder+GAN.
[0029] Preferred: The MUNIT network structure model module structure is as follows:
[0030] In step 5a), the content encoder extracts the content code from the input image data, uses several distributed convolutional layers to down sample the input image data, takes instance normalization, and after downsampling, inputs it into several residual blocks to obtain the content code.
[0031] In step 5b), the Style encoder extracts the style code from the input image data, performs downsampling operations through several convolutional layers, performs global average fusion operations after downsampling, and finally passes through a fully connected layer to extract the style information code.
[0032] In step 5c), the generator reconstructs the image based on the content code and style code. A series of residual blocks are used to process the content code. The residual blocks use Adaptive Instance Normalization layers, whose parameters (γ, β) are dynamically generated by performing an MLP multi-layer perceptron on the style code.
[0033]
[0034] Among them, z is the activation value generated by the previous convolutional layer, μ and σ are the channel mean and standard deviation, and γ and β are the parameters generated by the MLP from the style code. After residual processing, several upsampling layers and convolutional layers generate the reconstructed image, using ReLU and tanh as activation functions.
[0035] In step 5d), the discriminator uses a multi-scale discriminant network to downsample the real samples and generated samples to obtain images of different resolutions, and uses the discriminant model for identification to speed up the training of the model.
[0036] In step 5e), the least squares loss is used instead of the cross entropy loss as the objective function. Using LSGAN, the quality of the generated images is increased and the stability of the training is also improved.
[0037] Preferably, in step 5d), the activation function of the discriminant model is selected as the lrelu function.
[0038] Compared with the prior art, the present invention has the following beneficial effects:
[0039] Based on the task release scenario in crowd testing, the present invention generates a fluent task introduction text through structured key task words based on rule templates, deep learning generative adversarial networks and other related technologies; generates a word cloud through task text description and related pictures; uses the images provided by the task publisher as source domain images and related style target domain images to train the model and complete the corresponding style transfer. Based on the above technologies, the present invention realizes a more intelligent and interactive task release process. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 It is the overall flow chart of the method of the present invention;
[0041] Figure 2 Generate task text description, word cloud diagram and style image flow chart for the present invention.
[0042] Figure 3 This is a model diagram of the image style transfer process of the present invention. DETAILED DESCRIPTION
[0043] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.
[0044] A graphic and text generation technology for crowd testing task release, such as Figure 1-3 As shown, the following steps are included:
[0045] Step 1: Data collection. Obtain enough relevant data from real crowd-testing scenario task releases, including structured task key fields, detailed descriptions of task releases, and relevant content images or logo images provided by the task publisher, which are used for structured task release text generation and task release image generation (including word cloud images and relevant task style images).
[0046] Step 2, data preprocessing, mainly involves data organization and cleaning. The process is as follows:
[0047] Step 2a) processing structured keywords required to generate a fluent task text description;
[0048] Step 2b) Perform relevant statistics on the text descriptions required for word cloud image generation, clean them, and remove unreasonable parts;
[0049] Step 2c) performing correlation processing on the source domain image and the target domain image required for style transfer image generation. The target domain image can be obtained from data sets such as oil painting style images, emoji images, and anime dimensional images, or from related crawlers;
[0050] Step 2d) saving the processed data or data set and using it in subsequent tasks;
[0051] Step 3: Generate a fluent task text description. Based on the structured task-related words preprocessed in step 2, use a rule-based and template-based method to generate a high-quality and complete task text description. The method is as follows:
[0052] Step 3a) Count the key fields of the structured task description, including task name, description information, task type, task reward, deadline, number of people required, etc. (for example: 'entity':'Android app Economic Daily bug exploration', 'desc':'Bug exploration of app', 'type':'Functional testing', 'reward':50000, 'ddl':'October 31, 2020', 'peopleNum':50, 'taskNum':5}) as the main input information for subsequent task text generation.
[0053] Step 3b) Use expert intervention to set relevant rules and templates. The main content of the template includes relevant emotional words (such as "Don't hesitate, first come first served!", "You can make extra money by staying at home!"), some humorous words, connecting word sequences, etc. In addition, it is also necessary to consider the focus of the entire template, such as whether the task has special requirements for remuneration, type, number of people, or task deadline. And perform secondary processing on the template at a specific level.
[0054] Step 3c) Generate several text messages based on the input key fields using relevant expert rules and template strategies. The focus of this link is to select appropriate templates according to different tasks, to ensure that the content is introduced appropriately while increasing the fluency of the entire text description, such as the connection of relevant humorous words and emotional words; by extracting relevant task keywords, determine the overall style of the entire task description, whether there is a focus, and improve the text output of the entire task release.
[0055] Step 4: Generate a word cloud diagram for task release. Generate a word cloud diagram related to the task based on the task description provided by the relevant task party and related background images such as logo images. The specific method is as follows:
[0056] Step 4a) Obtain the task description provided by the relevant task party and perform relevant processing. Perform relevant Chinese word segmentation on the task description, set stop words, and perform word frequency statistics.
[0057] The task description here is different from the text description in step 2. The text description in step 2 integrates and optimizes the relevant key points of the entire task release process, focusing on the task name, salary, deadline, etc. The text description used in this step is an introduction to the specific details of the task. For example, if the task released is a classification task, the description may be "This project requires you to first classify the target sentence, and then extract and classify the target sentence keywords...", that is, the part about the specific implementation details of the task;
[0058] Step 4b) Process the relevant task images provided by the task party, such as the logo image, as the background template for the word cloud generation image. First, determine the ratio of white area to color area in the image, and select the appropriate area as the filling part of the word cloud. Further determine the relevant color conversion threshold, and convert the input image into a binary image; the binary image is the input background for the subsequent word cloud visualization.
[0059] Step 4c) Use the word cloud visualization method to set the relevant font, color, and number of displayed words, use the processed image in step 4b) as the word cloud background, and use the relevant task description word segmentation results in step 4a) as the word cloud content;
[0060] Step 4d) Dynamically adjust the generation effect of the word cloud map, visually adjust the color distribution, adjust the number and importance of related words, merge unnecessary repetitions of related words, and make the entire visual word cloud map more concise and reasonable.
[0061] Step 5: Generate relevant style images for the task release. Use the relevant data sets obtained by statistics, including source images - relevant pictures or logo images provided by the publisher, and relevant images of the target style - such as humorous style, stick figure, and oil painting style image data sets. Use the MUNIT network model in the generative adversarial network to perform unsupervised training of the style transfer model. The style transfer image task is abstracted to achieve multimodal image translation under the condition of unpaired data sets, and select the appropriate model MUNIT. The specific method is as follows:
[0062] Step 5a) The task can be abstractly summarized as using GAN to achieve multimodal image translation under unpaired dataset conditions. The MUNIT network structure model is adopted. The architecture of the model is auto-encoder+GAN. The core part of the model implementation lies in the encoder (including content encoder and style encoder), decoder / generator (decoder) and discriminator (discriminator). The following mainly explains the implementation of the model through several key components:
[0063] Step 5a1) The image translation process is assumed to be divided into a content space shared between potential domains and a style space determined independently by the domains. The content space encodes the complex data space structure, and the style space renders it. This process is implemented by two encoders E1 and E2, followed by a generator and Perform reorganization and decoding work to generate images of the corresponding domain.
[0064] Step 5a2) The main task of the content encoder is to extract the content code from the input image data. In our setting, the content code belongs to a shared space, which is a high-dimensional space mapping and has a more complex distribution rather than a Gaussian normal distribution.
[0065] Step 5a3) The main task of the Style encoder is to extract style codes from the input image data. Since the style code is Domain-independent and mainly renders the corresponding structure (content), it has a global and relatively simple impact. Therefore, we use a low-dimensional vector representation modeled by Gaussian prior.
[0066] Step 5a4) The generator reconstructs the image based on the content and style code. We use a series of residual blocks to process the content encoding. The residual blocks use Adaptive Instance Normalization layers, whose parameters (γ and β) are dynamically generated by performing an MLP multi-layer perceptron on the style code.
[0067] Step 5a5) The discriminator uses a multi-scale discriminant network, down-samples the real samples and the generated samples to obtain images of different resolutions, and uses the discriminant model for identification, which can speed up the training of the model. The activation function of the discriminant model is lrelu.
[0068] Step 5b) Loss function setting. The loss function of the entire adversarial network consists of two parts: reconstruction loss function and GAN loss function. The former ensures that the encoder and generator are reversible; the latter adopts style-enhanced cycle consistency, which is a weaker form of cycle consistency between the joint space of image styles. The reconstruction part draws on cycle-consistency to ensure that the encoder and generator are reversible. It mainly consists of two parts. One part is the in-domain image reconstruction, where the image is encoded into the latent space and then decoded to return to the original image; the other part is the latent space reconstruction, that is, the content space and style space can be encoded back to the original space after decoding (generator). Use the L1 norm to measure, where q(s) is the Gaussian distribution N(0,I); p(c_1) comes from
[0069] Step 5c) The model borrows the cycle consistency constraint from CycleGAN, and MUNIT also makes improvements on this basis, because the image translation in the many-to-many mode has too strong constraints on the entire space, which affects the final generation effect. Therefore, the model adopts style-enhanced cycle consistency, which is a weaker form of cycle consistency between the joint spaces of image styles.
[0070] Step 5d) In the image generation part, auto-encoders and GAN are combined to complete it. The training goal is to learn the underlying encoder and generator to complete the entire translation process. It should be noted that the potential distribution in the encoding process needs to match the potential distribution received by the decoder in the generation process, usually using KL divergence or adversarial loss. If the decoder receives a very different potential distribution during the generation process, the entire model will lose the generation effect. When the optimality of the model is reached, p(c1)=p(c2), p(s1)=q(s1), p(s2)=q(s2), at this time the encoding style distribution matches the Gaussian prior, and the encoding content distribution matches the distribution at the time of generation. The training goal is to learn the underlying encoder and generator to complete the entire translation process.
[0071] Step 5e) completes the style transfer from the source image provided by the task publisher to the target domain image by training the entire generation model, while retaining the content of the source image; making the published image more visual and style-featured.
[0072] In summary, the present invention proposes a graphic and text generation technology for crowd-testing task release, which generates corresponding task release text and related task pictures, which, on the one hand, reduces the pressure on the crowd-testing platform management party, and on the other hand, can promote interaction with people using the crowd-testing platform, and the intelligent method can highlight relevant key points, facilitate quick viewing of tasks and save time. The present invention generates corresponding task release text and related task pictures, which, while reducing the pressure on the crowd-testing platform management party, can increase the interaction of people using the crowd-testing platform and make the key content of related tasks more prominent, thereby improving the efficiency of browsing and viewing tasks.
[0073] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for generating images and texts for crowd-testing task releases. It is characterized in that The steps include: Step 1, data collection: obtain relevant data in the task release of the crowd testing scenario, including structured task key fields, detailed description of the task release, and relevant content pictures or logo pictures provided by the task publisher; used for structured task release text generation and task release picture generation, respectively. Task release picture generation includes word cloud pictures and relevant task style migration pictures; Step 2: Data preprocessing: sorting and cleaning the data for different tasks; Step 3: Generate task text description: Generate high-quality and complete task text description using rule-based and template-based methods based on the preprocessed structured task key fields: Step 3a), count the key fields of structured tasks, including task name, description, task type, task reward, deadline, and number of people required; Step 3b), using expert intervention to set relevant rules and templates, the main content of the template includes relevant emotional words, some humorous words, and connecting word sequences; the focus of the template is determined according to different tasks; Step 3c), based on the input structured task key fields, use relevant expert rules and template strategies to generate several text messages; the focus of this link is to select appropriate templates according to different tasks, to ensure that the content introduction is appropriate and the fluency of the entire text description can be increased; by extracting relevant task keywords, determine the overall style of the entire task description, whether there is a focus, and improve the text output of the entire task release; Step 4, generate a word cloud map: generate a word cloud map related to the task according to the task description and related content map or logo image provided by the relevant task party; On the basis of generating word cloud diagram, a dynamic modification method is added to visually adjust the color distribution, adjust the number and importance of related words, and merge repeated related words to make the entire visual word cloud diagram more concise and reasonable: Step 4a), perform relevant Chinese word segmentation on the task description provided by the relevant task party, set stop words, and perform word frequency statistics; Step 4b), process the relevant task pictures or logo pictures provided by the task party, and select a suitable area as the filling background template of the word cloud; Step 4c), using the word cloud visualization method, setting the relevant font, color, and number of displayed words, using the processed image obtained in step 4b) as the word cloud background, and using the relevant task description word segmentation results in step 4a) as the word cloud content; Step 4d), dynamically adjust the generation effect of the word cloud map, visually adjust the color distribution, adjust the number and importance of related words, merge repeated related words, so that the entire visual word cloud map is more concise and reasonable; Step 5: Generate relevant style images, use the statistically obtained relevant style dataset, and use the MUNIT network model in the generative adversarial network to perform unsupervised training of the style transfer model.
2. According to the method for generating images and texts for crowd-testing task release according to claim 1, Features: Data preprocessing method in step 2: Step 2a), processing structured keywords required to generate a fluent task text description; Step 2b), perform relevant statistics on the text descriptions required for word cloud image generation, clean them, and remove unreasonable parts; Step 2c), performing correlation processing on the source domain image and the target domain image required for style transfer map generation, wherein the target domain image is obtained from a dataset of oil painting style images, emoji images, and anime dimensional images or a related crawler; Step 2d), save the processed data or data set and use it in subsequent tasks.
3. According to the method for generating images and texts for crowd-testing task release according to claim 2, Features: The data format of the structured task keywords in step 3 is {'entity, 'description', 'type', 'salary', 'deadline', 'number of people required', 'number of tasks'}; the structured task keywords are used as input into the rule template to generate a suitable task description.
4. According to the method for generating images and texts for crowd-testing task release according to claim 3, Features: The step 5 of generating relevant style images can be abstractly summarized as using GAN to achieve multimodal image translation under unpaired data set conditions; using the MUNIT network structure model, and the MUNIT network structure model architecture is auto-encoder+GAN.
5. According to the method for generating images and texts for crowd-testing task release according to claim 4, Features: The module structure of the MUNIT network structure model is as follows: Step 5a), the content encoder extracts the content code from the input image data, uses several distributed convolutional layers to down sample the input image data, uses instance_normalization to perform instance normalization, and after downsampling, inputs it into several residual blocks to obtain the content code; Step 5b), the Style encoder extracts the style code from the input image data, performs downsampling operations through several convolutional layers, performs global average convergence operations after downsampling, and finally passes through a fully connected layer to extract the style information code; Step 5c), the generator reconstructs the image based on the content code and style code; a series of residual blocks are used to process the content coding. The residual blocks use Adaptive Instance Normalization layers, whose parameters (γ, β) are dynamically generated by performing an MLP multi-layer perceptron on the style code; Among them, z is the activation value generated by the previous convolutional layer, μ and σ are the channel mean and standard deviation, γ and β are the parameters generated by MLP from stylecode; after residual processing, the reconstructed image is generated by several upsampling layers and convolutional layers, using ReLU and tanh as activation functions; In step 5d), the discriminator uses a multi-scale discriminant network to downsample the real samples and generated samples to obtain images of different resolutions, and uses the discriminant model for identification to speed up the training of the model; In step 5e), the least square loss is used instead of the cross entropy loss as the objective function; using LSGAN, the quality of the generated images is increased and the stability of the training is also improved.
6. According to the method for generating images and texts for crowd-testing task release according to claim 5, Features: In step 5d), the activation function of the discriminant model is selected as the lrelu function.
