Test paper image generation method and device, storage medium and electronic equipment

Abstract

The present disclosure relates to a test paper image generation method and device, a storage medium and an electronic device, and belongs to the technical field of information. The method comprises: generating a test paper layout based on a pre-acquired material library, and adding test questions based on the test paper layout to obtain a first test paper image with multiple annotation information, wherein each material in the material library includes at least one annotation information; inputting the first test paper image into any one of a plurality of pre-trained test paper image generation models to obtain a target test paper image with multiple annotation information after style transfer, so that the target test question image presents an image effect of real shooting by a user, wherein the style transfer types of each test paper image generation model are different. The number and diversity of training data are expanded, the annotation cost is reduced, and the efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of information technology, and in particular to a method, device, storage medium and electronic device for generating an examination paper image. Background Art

[0002] Deep learning-based text recognition and layout analysis of common test paper images in photographic settings requires data support. Satisfactory results require at least tens of thousands of images. Collecting and labeling these images is a time-consuming and labor-intensive process. Even with sufficient numbers of test paper images, labeling them often requires manual labor. However, obtaining tens of thousands of labeled images is impossible. Summary of the Invention

[0003] In order to solve the problems existing in the related art, the present disclosure provides a test paper image generation method, device, storage medium and electronic device.

[0004] In order to achieve the above-mentioned object, the present disclosure provides a first aspect of a method for generating an examination paper image, the method comprising:

[0005] Generating a test paper layout based on a pre-acquired material library, and adding test questions based on the test paper layout to obtain a first test paper image including a plurality of annotation information, wherein each material in the material library includes at least one type of annotation information;

[0006] The first test paper image is input into any one of the multiple test paper image generation models obtained in advance, and a target test paper image with multiple annotation information is obtained after style transfer, so that the target test paper image presents the image effect of a real photo taken by the user, wherein the style transfer type of each test paper image generation model is different.

[0007] Optionally, the method comprises:

[0008] Obtain a public dataset with annotations as a training set, pre-train the initial semantic segmentation model, and obtain the first trained semantic segmentation model;

[0009] Obtaining a test paper image set without annotated information, and annotating a target number of test papers in the test paper image set based on the annotated categories of the public dataset, to obtain a first annotated image set and a second image set without annotated information, such that the annotated categories of the annotated first image set are consistent with the annotated categories of the public dataset, and the number of test paper images in the first image set is less than that in the second image set;

[0010] Inputting the first image set into the first semantic segmentation model to perform transfer learning to obtain a second semantic segmentation model;

[0011] The second image set is input into the second semantic segmentation model, and semantic segmentation is performed on the second image set to obtain the material library, where each material in the material library includes annotation information corresponding to the material.

[0012] Optionally, generating a test paper layout based on the material library, and adding test questions to the test paper layout to obtain a first test paper image includes:

[0013] Determine the size information of the test paper image to be generated, wherein the size information includes width, height and number of columns;

[0014] Based on the size information and the materials in the material library, a test paper layout is generated, wherein the test paper layout includes layout information and test paper title information, wherein the layout information is used to represent the division of each area of ​​the test paper image and the boundary information of each area; and

[0015] Randomly selecting target question types from each area of ​​the test paper image to be generated;

[0016] For each of the areas, question corpus corresponding to the target question type is selected from the material library, and the question corpus is added to the area according to the boundary information of the area represented by the layout information, until each area is full, so as to obtain the first test paper image.

[0017] Optionally, the training of the test paper image generation model includes:

[0018] Obtaining a scanned image set obtained by scanning, and a captured image set corresponding to each test paper image in the scanned image set and captured in a manner corresponding to the target style transfer type;

[0019] Inputting the scanned image set into a first generator to obtain a first generated image set;

[0020] Inputting the first generated image set and the captured image set into a first discriminator to obtain a first discrimination result;

[0021] Adjusting the parameters of the first generator according to the first discrimination result to obtain an updated first generator;

[0022] Repeat the above steps of inputting the scanned image set into the first generator to adjusting the parameters of the first generator until the first discrimination result indicates that the confidence of the first generated image set generated by the first generator is greater than a preset threshold, and use the first generator obtained by the last update as the test paper image generation model corresponding to the target style transfer type.

[0023] Optionally, any one of the test paper image generation models includes N+1 generators connected in series, where N is a natural number greater than or equal to zero.

[0024] Inputting the first test paper image into any one of a plurality of pre-trained test paper image generation models to obtain a target test paper image with multiple annotated information after style transfer includes:

[0025] For the first generator among the N+1 generators connected in series, inputting the first test paper image that has been downsampled N times into the first generator to obtain a generated image corresponding to the first generator;

[0026] For the Mth generator among the N+1 generators connected in series, an image obtained by upsampling the generated image output by the M-1th generator and the first test paper image that has been downsampled N-M+1 times are input into the Mth generator to obtain a generated image corresponding to the Mth generator, where M is greater than or equal to 2 and less than or equal to N+1;

[0027] The generated image output by the Nth generator is used as the target test paper image.

[0028] Optionally, the training of the test paper image generation model includes:

[0029] Get sample test images of the target style transfer type;

[0030] Inputting the sample test paper image into an initial test paper image generation model, wherein the initial test paper image generation model comprises N+1 untrained generators connected in series, wherein each untrained generator is connected to a discriminator;

[0031] For the first generator among the N+1 untrained generators connected in series, random noise is input into the first generator to obtain a generated image corresponding to the first generator; the generated image corresponding to the first generator and a sample test paper image that has been downsampled N times are input into the discriminator corresponding to the first generator to obtain a discrimination result output by the discriminator corresponding to the first generator, and the parameters of the first generator are adjusted according to the discrimination result to obtain a trained first generator;

[0032] For the Mth generator among the N+1 untrained generators connected in series, the random noise that has been upsampled M-1 times and the generated image output by the trained M-1th generator are input into the Mth generator to obtain the generated image corresponding to the Mth generator; the generated image corresponding to the Mth generator and the sample test paper image that has been downsampled N-M+1 times are input into the discriminator corresponding to the Mth generator to obtain the discrimination result output by the discriminator corresponding to the Mth generator, and the parameters of the Mth generator are adjusted according to the discrimination result to obtain the trained Mth generator, where M is greater than or equal to 2 and less than or equal to N+1;

[0033] According to the trained 1st generator to the N+1th generator, a trained test paper image generation model corresponding to the target style transfer type is obtained.

[0034] Optionally, the generator and the discriminator are both composed of five convolution blocks, each convolution block includes a convolution layer, a batch normalization layer and an activation layer, and the generator is composed of a fully convolutional network.

[0035] A second aspect of the present disclosure provides a test paper image generating device, the device comprising:

[0036] a first generating module configured to generate a test paper layout based on a pre-acquired material library, and to add test questions based on the test paper layout to obtain a first test paper image including a plurality of annotation information, wherein each material in the material library includes at least one type of annotation information;

[0037] The second generation module is used to input the first test paper image into any one of the multiple test paper image generation models obtained in advance, and obtain a target test paper image with multiple annotation information after style transfer, so that the target test paper image presents the image effect of a real photo taken by the user, wherein the style transfer type of each of the test paper image generation models is different.

[0038] A third aspect of the present disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect of the present disclosure.

[0039] A fourth aspect of the present disclosure provides an electronic device, including:

[0040] a memory having a computer program stored thereon;

[0041] A processor is used to execute the computer program in the memory to implement the steps of any one of the methods described in the first aspect of the present disclosure.

[0042] Through the above technical solution, an electronic test paper is generated through a pre-acquired material library with annotated information, and the electronic test paper is input into a pre-trained test paper image generation model corresponding to various style transfer types, thereby obtaining a test paper image that presents the image effect of a real user photo. This enables the generation of a large number of test paper images with the image effect of a real user photo, expands the quantity and diversity of training data, reduces annotation costs, and improves efficiency.

[0043] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0045] Figure 1 is a flow chart showing a method for generating a test paper image according to an exemplary embodiment;

[0046] Figure 2 is a flow chart showing a method for obtaining a material library according to an exemplary embodiment;

[0047] Figure 3 is a schematic diagram of a test paper image generation model according to an exemplary embodiment;

[0048] Figure 4 is a block diagram of a test paper image generating device according to an exemplary embodiment;

[0049] Figure 5 is a block diagram of an electronic device according to an exemplary embodiment;

[0050] Figure 6 is another block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0051] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.

[0052] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the corresponding data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0053] In order to enable those skilled in the art to better understand the improvements of the technical solution proposed in this disclosure, this disclosure introduces the related technologies. In the related technologies, there are mainly four solutions to the missing test paper data:

[0054] 1. Use public datasets with annotations for pre-training.

[0055] 2. Collect data, label it, and then train the model.

[0056] 3. Synthesize text line data for text detection and recognition.

[0057] 4. Generate electronic test paper.

[0058] For the first solution, since public datasets are used, the annotation content of these datasets may be different, so only one can be selected for use. In addition, the application scenarios of public datasets cannot be unified with the requirements of general test paper image text detection, recognition and layout analysis in this proposal, and cannot be used as a supplement to the data.

[0059] For the second solution, it is time-consuming and labor-intensive to collect general test paper images of various styles of photo scenes. In addition, the cost of comprehensive annotation of the images is very high.

[0060] For the third solution, text lines are generated by synthesizing data, which can be used for text detection and recognition. However, due to the lack of a photo-taking scene for the test paper, its effectiveness is limited. In addition, the test paper layout information is missing, and the generated image does not imitate the style of the real test paper as much as possible, which is far from enough for text recognition and layout analysis of the test paper.

[0061] The fourth solution focuses on automatically generating test papers for students, rather than for deep learning model training. First, the generated test papers have a relatively simple format, lack comprehensive coverage of subject matter, and lack corresponding labels for the test content. Second, the generated electronic test paper images differ significantly from the actual photographed test paper images, making them unsuitable for use in these scenarios.

[0062] In order to solve the problems existing in the related art, the present disclosure provides a test paper image generation method, device, storage medium and electronic device.

[0063] Figure 1 This is a flow chart of a method for generating a test paper image according to an exemplary embodiment. The execution subject of this method can be a notebook computer, a mobile phone, a server or other electronic device with information processing capabilities. This disclosure does not specifically limit this. Figure 1 As shown, the method includes:

[0064] S101. Generate a test paper layout based on a pre-acquired material library, and add test questions based on the test paper layout to obtain a first test paper image including multiple annotation information, wherein each material in the material library includes at least one annotation information.

[0065] It can be understood that each material in the material library includes at least one corresponding annotation information. For example, there is a question in the material library, which may include the corresponding question area, question type, question stem, answer, question number, page number, question inclusion relationship, picture, table, formula, option and other annotation information, or the material library also includes a test paper title, which includes corresponding boundary information, text information and other annotations.

[0066] Among them, the test paper layout may include, for example, column information, headline information, sealing line information and attribute information. The column information may include the boundary information of each area of ​​the test paper, the headline information includes the headline text and the boundary information corresponding to the headline, the sealing line information includes the boundary information of the sealing line in the test paper, and the attribute information may include prompts for students to write their names, classes and other attributes, as well as the boundary information of the writing lines.

[0067] It is understandable that the above-mentioned first test paper images may be multiple, for example, 100,000 or more.

[0068] S102. Input the first test paper image into any one of the multiple test paper image generation models obtained in advance, and obtain a target test paper image with multiple annotation information after style transfer, so that the target test paper image presents the image effect of a real photo taken by the user, wherein the style transfer type of each of the test paper image generation models is different.

[0069] Among them, the image effects presented as real shots by the user may include, for example, light changes, paper wrinkles, colors, watermarks, blur, etc. The above style transfer types may correspond to the image effects of each real shot.

[0070] Moreover, based on the various annotation information of the above-mentioned target test paper image, the target test paper image can be applied to the detection and recognition of handwritten and printed text in test paper images, detection and recognition of graphic tables in test paper images, detection and recognition of formulas in test paper images, detection and recognition of question numbers, detection and recognition of page numbers, column detection, detection of question blocks in test paper images, relationship recognition, named entity recognition, intelligent grading of test paper images, and other fields.

[0071] After obtaining the target test question image based on the above method, the target test question image can be used to train a deep learning model for test paper text recognition and layout analysis.

[0072] In the disclosed embodiment, an electronic test paper is generated by using a pre-acquired material library with annotated information, and the electronic test paper is input into a pre-trained test paper image generation model corresponding to various style transfer types, thereby obtaining a test paper image that presents the image effect of a real user photo. This enables the generation of a large number of test paper images that present the image effect of a real user photo, thereby expanding the quantity and diversity of training data, reducing annotation costs, and improving efficiency.

[0073] In a possible implementation, before step S101, the material library may be obtained by Figure 2 Obtained in the manner shown, refer to Figure 2 , the method further comprises:

[0074] S201: Obtain a public dataset with annotations as a training set, pre-train an initial semantic segmentation model, and obtain a trained first semantic segmentation model.

[0075] The initial semantic segmentation model can be a Mask R-CNN model, and the annotated public dataset can be the annotated PubLayNet public dataset, which is a document image layout analysis dataset. The first semantic segmentation model obtained by training on the above public dataset can achieve semantic segmentation of document images.

[0076] S202. Obtain a test paper image set without labeled information, and label a target number of test papers in the test paper image set based on the labeling categories of the public data set to obtain a first labeled image set and a second image set without labeled information, so that the labeling categories of the first labeled image set are consistent with the labeling categories of the public data set.

[0077] The number of test paper images in the first image set is smaller than that in the second image set.

[0078] It should be understood that the test paper image set without labeled information may be a set of photographed images obtained by photographing the test paper.

[0079] Among them, the specific numerical value of the above-mentioned target number can be directly calibrated or determined according to a preset ratio. For example, if the set of test paper images without annotation information includes 100 test paper images, the test paper images without annotation information can be annotated at a ratio of 5%, that is, 5 test paper images are selected for annotation.

[0080] S203: Input the first image set into the first semantic segmentation model to perform transfer learning to obtain a second semantic segmentation model.

[0081] Those skilled in the art should be aware that the transfer learning recorded in step S201 and step S203 and the pre-training process for the initial semantic segmentation model may involve multiple iterations, and based on multiple iterations, after the loss value representation of the semantic segmentation model after the current iteration meets the preset conditions, the iteration is stopped to obtain the above-mentioned pre-trained first semantic segmentation model and the second semantic segmentation model.

[0082] After step S202 and step S203, the second semantic segmentation model after transfer learning can perform semantic segmentation on the captured test paper image to extract text, images, tables and other areas in the test paper image.

[0083] Optionally, in the training of the second semantic segmentation model in step S203, a portion of the images in the second image set can be input into the model, the output results are analyzed, and images with high confidence are selected to be added to the next round of model training. After multiple iterative training, the final model is obtained.

[0084] S204: Input the second image set into the second semantic segmentation model, perform semantic segmentation on the second image set, and obtain the material library.

[0085] Each material in the material library includes annotation information corresponding to the material.

[0086] Among them, the annotation information may include the above-mentioned test paper layout information, and may also include question block information. The question block information may further include question area, question type, question stem, answer, question number, page number, question inclusion relationship, pictures, tables, formulas, options and other information.

[0087] By adopting the above scheme, the semantic segmentation model is pre-trained based on the public dataset, and only a small number of test images are annotated for semi-supervised learning, so as to obtain a model that can perform semantic segmentation on the captured images. The test image set without annotated information is then input into the model to extract the various materials in the image set and the corresponding annotation information to form a material library.

[0088] In some optional implementations, generating a test paper layout based on the material library, and adding test questions to the test paper layout to obtain a first test paper image includes:

[0089] Determine the size information of the test paper image to be generated, wherein the size information includes width, height, and number of columns;

[0090] Based on the size information and the materials in the material library, a test paper layout is generated, wherein the test paper layout includes layout information and test paper title information, wherein the layout information is used to represent the division of each area of ​​the test paper image and the boundary information of each area; and

[0091] Randomly selecting target question types from each area of ​​the test paper image to be generated;

[0092] For each of the areas, question corpus corresponding to the target question type is selected from the material library, and the question corpus is added to the area according to the boundary information of the area represented by the layout information, until each area is full, so as to obtain the first test paper image.

[0093] Specifically, for the first test paper image to be synthesized, its length, width, and number of columns are determined, and then two modules of synthesis layout and synthesis question are carried out. The synthesis layout can be obtained by synthesizing the contents of the upper, lower, left, and right column dividing lines of the image through the image width, height, and number of columns, and the synthesized material is provided by the previous method; the synthesis question can be obtained by determining the upper, lower, left, and right boundaries of the column area, and writing the question from top to bottom in the area. Each time a question type is written, a question type is randomly selected, and then according to the characteristics of the question type, the text and charts in the corpus are written into the corresponding area of ​​the image, and the corresponding annotation content is written into the annotation file until the area is completed, and finally the first test paper image is generated.

[0094] By adopting this solution, the size information of the first test paper image to be generated is first determined, and based on the size information, the test paper layout with layout information is determined. Then, based on the layout information, questions in the material library are filled in each area according to the question type, thereby obtaining a first test paper image that is closer to the real test paper.

[0095] In one possible implementation, the training of the test paper image generation model includes:

[0096] Obtaining a scanned image set obtained by scanning, and a captured image set corresponding to each test paper image in the scanned image set and captured in a manner corresponding to the target style transfer type;

[0097] Inputting the scanned image set into a first generator to obtain a first generated image set;

[0098] Inputting the first generated image set and the captured image set into a first discriminator to obtain a first discrimination result;

[0099] Adjusting the parameters of the first generator according to the first discrimination result to obtain an updated first generator;

[0100] Repeat the above steps of inputting the scanned image set into the first generator to adjusting the parameters of the first generator until the first discrimination result indicates that the confidence of the first generated image set generated by the first generator is greater than a preset threshold, and use the first generator obtained by the last update as the test paper image generation model corresponding to the target style transfer type.

[0101] The above-mentioned scanned image set may also be an electronic document image set of a test paper, and the characteristics of the image set are that it is clear, easy to identify, and difficult to present the effect of an image actually taken by a user.

[0102] Optionally, the above-mentioned first generator and first discriminator can both be composed of five convolution blocks, each convolution block includes a convolution layer, a batch normalization layer and an activation layer, and the first generator can be composed of a fully convolutional network.

[0103] This solution is adopted to obtain a scanned image of the test paper and a photographed image of a specific style transfer type corresponding to the test paper based on a generative adversarial network. The scanned image is input into the generator, and the image generated by the generator is judged based on the photographed image after style transfer, and then the parameters of the generator are adjusted so that the generator can more accurately generate a style-transferred image based on an image that is difficult to present as a real photo taken by the user, so that the target test paper image generated based on the first test paper image can more accurately present the image effect of a real photo taken by the user.

[0104] In some other possible implementations, any one of the test paper image generation models includes N+1 generators connected in series, where N is a natural number greater than or equal to zero.

[0105] Inputting the first test paper image into any one of a plurality of pre-trained test paper image generation models to obtain a target test paper image with multiple annotated information after style transfer includes:

[0106] For the first generator among the N+1 generators connected in series, inputting the first test paper image that has been downsampled N times into the first generator to obtain a generated image corresponding to the first generator;

[0107] For the Mth generator among the N+1 generators connected in series, an image obtained by upsampling the generated image output by the M-1th generator and the first test paper image that has been downsampled N-M+1 times are input into the Mth generator to obtain a generated image corresponding to the Mth generator, where M is greater than or equal to 2 and less than or equal to N+1;

[0108] The generated image output by the Nth generator is used as the target test paper image.

[0109] Among them, the multiples of downsampling and upsampling are the same, and the specific values ​​can be determined according to the actual model training effect, which is not limited in this disclosure.

[0110] Using the above scheme, the synthetic image is downsampled and used as the model input to generate the corresponding picture style result, in which the content and position in the image do not change, and the texture and high-frequency information will match the image of this style transfer type.

[0111] Optionally, the training of the test paper image generation model includes:

[0112] Get sample test images of the target style transfer type;

[0113] Inputting the sample test paper image into an initial test paper image generation model, wherein the initial test paper image generation model comprises N+1 untrained generators connected in series, wherein each untrained generator is connected to a discriminator;

[0114] For the first generator among the N+1 untrained generators connected in series, random noise is input into the first generator to obtain a generated image corresponding to the first generator; the generated image corresponding to the first generator and a sample test paper image that has been downsampled N times are input into the discriminator corresponding to the first generator to obtain a discrimination result output by the discriminator corresponding to the first generator, and the parameters of the first generator are adjusted according to the discrimination result to obtain a trained first generator;

[0115] For the Mth generator among the N+1 untrained generators connected in series, the random noise that has been upsampled M-1 times and the generated image output by the trained M-1th generator are input into the Mth generator to obtain the generated image corresponding to the Mth generator; the generated image corresponding to the Mth generator and the sample test paper image that has been downsampled N-M+1 times are input into the discriminator corresponding to the Mth generator to obtain the discrimination result output by the discriminator corresponding to the Mth generator, and the parameters of the Mth generator are adjusted according to the discrimination result to obtain the trained Mth generator, where M is greater than or equal to 2 and less than or equal to N+1;

[0116] According to the trained 1st generator to the N+1th generator, a trained test paper image generation model corresponding to the target style transfer type is obtained.

[0117] Similarly, the discriminators corresponding to the above N+1 generators connected in series can be composed of five convolution blocks, each of which includes a convolution layer, a batch normalization layer and an activation layer, and the first generator can be composed of a fully convolutional network.

[0118] In order to make those skilled in the art better understand the training method of the above-mentioned test paper image generation model, the present disclosure also provides the following Figure 3 A schematic diagram of a test paper image generation model is shown in Figure 3 As shown, the test paper image generation model includes N+1 serially connected generators, namely G(0) to G(N), and N+1 discriminators corresponding to the N+1 serially connected generators, namely D(0)-D(N).

[0119] Taking the untrained generator G(0) and discriminator D(0) at the bottom layer as an example, random noise Z(0) is used as the input of G(0) to obtain the output result X'(0). X'(0) and the real image X(N) of the corresponding target style transfer type that has been downsampled N times are input into the discriminator D(0) to obtain the discrimination result, and the parameters of G(0) are adjusted. Repeat the iteration multiple times until the parameters of G(0) and D(0) are fixed, and it is determined that the training of the generator G(0) is completed.

[0120] Furthermore, Z(0) is input into the trained generator G(0) to obtain the final output result X'(0), Z(0) is upsampled to obtain Z(1), and X'(1) and Z(1) are input into the untrained generator G(1) and discriminator D(1) of the previous layer. Moreover, based on the real image X(N-1) of the corresponding target style transfer type that has been downsampled N-1 times, the above training process of the generator G(0) and the discriminator D(0) is repeated to obtain the trained generator G(1) and discriminator D(1), and so on, to obtain the trained generator G(N) and discriminator D(N).

[0121] It should be understood that A in Z(A) represents the number of upsampling times, and B in X(B) represents the number of downsampling times of the actual image corresponding to the target style transfer type.

[0122] By removing the discriminator, the final trained test paper image generation model can be obtained, so that the test paper image generation model can generate images of the corresponding style transfer type based on the downsampled image as the model input, and ensure that the content and position in the image will not change, and the texture and high-frequency information will be consistent with the style transfer type corresponding to the model.

[0123] Figure 4 is a block diagram of a test paper image generating device 40 according to an exemplary embodiment. Figure 4 As shown, the device 40 includes:

[0124] A first generating module 41 is configured to generate a test paper layout based on a pre-acquired material library, and add test questions based on the test paper layout to obtain a first test paper image including a plurality of annotation information, wherein each material in the material library includes at least one type of annotation information;

[0125] The second generation module 42 is used to input the first test paper image into any one of the multiple test paper image generation models obtained in advance, and obtain a target test paper image with multiple annotation information after style transfer, so that the target test paper image presents the image effect of a real photo taken by the user, wherein the style transfer type of each of the test paper image generation models is different.

[0126] Optionally, the device 40 further includes:

[0127] A first acquisition module is used to obtain a public dataset with annotations as a training set, pre-train the initial semantic segmentation model, and obtain a trained first semantic segmentation model;

[0128] a second acquisition module for acquiring a set of test paper images without annotated information, and annotating a target number of test papers in the test paper image set based on the annotation categories of the public dataset, thereby obtaining a first annotated image set and a second image set without annotated information, such that the annotation categories of the first annotated image set are consistent with the annotation categories of the public dataset, and the number of test paper images in the first image set is smaller than that in the second image set;

[0129] a transfer learning module, configured to input the first image set into the first semantic segmentation model, perform transfer learning, and obtain a second semantic segmentation model;

[0130] The semantic segmentation module is used to input the second image set into the second semantic segmentation model, perform semantic segmentation on the second image set, and obtain the material library, wherein each material in the material library includes annotation information corresponding to the material.

[0131] Optionally, the first generating module 41 is specifically configured to include:

[0132] Determine the size information of the test paper image to be generated, wherein the size information includes width, height, and number of columns;

[0133] Based on the size information and the materials in the material library, a test paper layout is generated, wherein the test paper layout includes layout information and test paper title information, wherein the layout information is used to represent the division of each area of ​​the test paper image and the boundary information of each area; and

[0134] Randomly selecting target question types from each area of ​​the test paper image to be generated;

[0135] For each of the areas, question corpus corresponding to the target question type is selected from the material library, and the question corpus is added to the area according to the boundary information of the area represented by the layout information, until each area is full, so as to obtain the first test paper image.

[0136] Optionally, the device 40 further includes:

[0137] A third acquisition module is used to acquire a scanned image set obtained by scanning, and a captured image set corresponding to each test paper image in the scanned image set and captured in a manner corresponding to the target style transfer type;

[0138] A third generating module, configured to input the scanned image set into the first generator to obtain a first generated image set;

[0139] a first discrimination module, configured to input the first generated image set and the captured image set into a first discriminator to obtain a first discrimination result;

[0140] an updating module, configured to adjust parameters of the first generator according to the first discrimination result to obtain an updated first generator;

[0141] Repeat the above steps of inputting the scanned image set into the first generator to adjusting the parameters of the first generator until the first discrimination result indicates that the confidence of the first generated image set generated by the first generator is greater than a preset threshold, and use the first generator obtained by the last update as the test paper image generation model corresponding to the target style transfer type.

[0142] Optionally, any one of the test paper image generation models includes N+1 generators connected in series, where N is a natural number greater than or equal to zero.

[0143] The second generating module 42 is specifically configured to:

[0144] For the first generator among the N+1 generators connected in series, inputting the first test paper image that has been downsampled N times into the first generator to obtain a generated image corresponding to the first generator;

[0145] For the Mth generator among the N+1 generators connected in series, an image obtained by upsampling the generated image output by the M-1th generator and the first test paper image that has been downsampled N-M+1 times are input into the Mth generator to obtain a generated image corresponding to the Mth generator, where M is greater than or equal to 2 and less than or equal to N+1;

[0146] The generated image output by the Nth generator is used as the target test paper image.

[0147] Optionally, the device 40 further includes:

[0148] The fourth acquisition module is used to obtain a sample test paper image of the target style transfer type;

[0149] An input module, configured to input the sample test paper image into an initial test paper image generation model, wherein the initial test paper image generation model comprises N+1 untrained generators connected in series, each of which is connected to a discriminator;

[0150] The first training module is configured to input random noise into the first generator of the N+1 untrained generators connected in series, thereby obtaining a generated image corresponding to the first generator; input the generated image corresponding to the first generator and a sample test paper image that has been downsampled N times into a discriminator corresponding to the first generator, thereby obtaining a discrimination result output by the discriminator corresponding to the first generator; and adjust the parameters of the first generator according to the discrimination result to obtain a trained first generator;

[0151] The second training module inputs the random noise upsampled M-1 times and the generated image output by the trained M-1 generator into the M-th generator to obtain the generated image corresponding to the M-th generator; inputs the generated image corresponding to the M-th generator and the sample test paper image downsampled N-M+1 times into the discriminator corresponding to the M-th generator to obtain the discrimination result output by the discriminator corresponding to the M-th generator, and adjusts the parameters of the M-th generator according to the discrimination result to obtain the trained M-th generator, where M is greater than or equal to 2 and less than or equal to N+1;

[0152] A determination module is used to obtain a trained test paper image generation model corresponding to the target style transfer type based on the trained 1st generator to the N+1th generator.

[0153] Optionally, the generator and the discriminator are both composed of five convolution blocks, each convolution block includes a convolution layer, a batch normalization layer and an activation layer, and the generator is composed of a fully convolutional network.

[0154] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0155] Figure 5 FIG. 5 is a block diagram of an electronic device 500 according to an exemplary embodiment. Figure 5 As shown, the electronic device 500 may include: a processor 501 , a memory 502 , and may further include one or more of a multimedia component 503 , an input / output (I / O) interface 504 , and a communication component 505 .

[0156] The processor 501 is used to control the overall operation of the electronic device 500 to complete all or part of the steps in the above-mentioned test paper image generation method. The memory 502 is used to store various types of data to support the operation of the electronic device 500. For example, these data may include instructions for any application or method operating on the electronic device 500, as well as application-related data, such as target test paper images, material libraries, contact data, sent and received messages, pictures, audio, video, etc. The memory 502 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The multimedia component 503 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 502 or transmitted via the communication component 505. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 504 provides an interface between the processor 501 and other interface modules, which may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 505 is used for wired or wireless communication between the electronic device 500 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more thereof, is not limited here. Therefore, the corresponding communication component 505 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0157] In an exemplary embodiment, the electronic device 500 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned test paper image generation method.

[0158] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the above-mentioned test paper image generation method. For example, the computer-readable storage medium may be the above-mentioned memory 502 including the program instructions. The above-mentioned program instructions may be executed by the processor 501 of the electronic device 500 to perform the above-mentioned test paper image generation method.

[0159] Figure 6 6 is a block diagram of an electronic device 600 according to an exemplary embodiment. For example, the electronic device 600 can be provided as a server. Figure 6 The electronic device 600 includes a processor 622, which may be one or more, and a memory 632 for storing a computer program executable by the processor 622. The computer program stored in the memory 632 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 622 may be configured to execute the computer program to perform the above-mentioned test paper image generation method.

[0160] In addition, the electronic device 600 may further include a power supply component 626 and a communication component 650. The power supply component 626 may be configured to perform power management of the electronic device 600, and the communication component 650 may be configured to implement communication of the electronic device 600, for example, wired or wireless communication. In addition, the electronic device 600 may further include an input / output (I / O) interface 658. The electronic device 600 may operate based on an operating system stored in the memory 632, such as Windows Server 2008. TM , Mac OSX TM , Unix TM , Linux TM etc.

[0161] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the above-mentioned test paper image generation method. For example, the non-transitory computer-readable storage medium may be the aforementioned memory 632 including the program instructions. The program instructions may be executed by the processor 622 of the electronic device 600 to perform the above-mentioned test paper image generation method.

[0162] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device, and the computer program has a code portion for executing the above-mentioned test paper image generation method when executed by the programmable device.

[0163] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the scope of protection of the present disclosure.

[0164] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0165] In addition, the various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A test paper image generation method, characterized in that: The method comprises: Generating a test paper layout based on a pre-acquired material library, and adding test questions based on the test paper layout to obtain a first test paper image including a plurality of annotation information, wherein each material in the material library includes at least one type of annotation information; Inputting the first test paper image into any one of a plurality of pre-trained test paper image generation models to obtain a target test paper image having multiple annotated information after style transfer, so that the target test paper image appears to be an image actually taken by a user, wherein each of the test paper image generation models has a different style transfer type; The method comprises: Obtain a public dataset with annotations as a training set, pre-train the initial semantic segmentation model, and obtain the first trained semantic segmentation model; Obtaining a test paper image set without annotated information, and annotating a target number of test papers in the test paper image set based on the annotated categories of the public dataset, to obtain a first annotated image set and a second image set without annotated information, such that the annotated categories of the annotated first image set are consistent with the annotated categories of the public dataset, and the number of test paper images in the first image set is less than that in the second image set; Inputting the first image set into the first semantic segmentation model to perform transfer learning to obtain a second semantic segmentation model; The second image set is input into the second semantic segmentation model, and semantic segmentation is performed on the second image set to obtain the material library, where each material in the material library includes annotation information corresponding to the material.

2. The method according to claim 1, characterized in that The step of generating a test paper layout based on the material library and adding test questions to the test paper layout to obtain a first test paper image includes: Determine the size information of the test paper image to be generated, wherein the size information includes width, height, and number of columns; Based on the size information and the materials in the material library, a test paper layout is generated, wherein the test paper layout includes layout information and test paper title information, wherein the layout information is used to represent the division of each area of ​​the test paper image and the boundary information of each area; and Randomly selecting target question types from each area of ​​the test paper image to be generated; For each of the areas, question corpus corresponding to the target question type is selected from the material library, and the question corpus is added to the area according to the boundary information of the area represented by the layout information, until each area is full, so as to obtain the first test paper image.

3. The method according to claim 1, characterized in that The training of the test paper image generation model includes: Obtaining a scanned image set obtained by scanning, and a captured image set corresponding to each test paper image in the scanned image set and captured in a manner corresponding to the target style transfer type; Inputting the scanned image set into a first generator to obtain a first generated image set; Inputting the first generated image set and the captured image set into a first discriminator to obtain a first discrimination result; Adjusting the parameters of the first generator according to the first discrimination result to obtain an updated first generator; Repeat the above steps of inputting the scanned image set into the first generator to adjusting the parameters of the first generator until the first discrimination result indicates that the confidence of the first generated image set generated by the first generator is greater than a preset threshold, and use the first generator obtained by the last update as the test paper image generation model corresponding to the target style transfer type.

4. The method according to claim 1, wherein Any one of the test paper image generation models includes N+1 generators connected in series, where N is a natural number greater than or equal to zero. Inputting the first test paper image into any one of a plurality of pre-trained test paper image generation models to obtain a target test paper image with multiple annotated information after style transfer includes: For the first generator among the N+1 generators connected in series, inputting the first test paper image that has been downsampled N times into the first generator to obtain a generated image corresponding to the first generator; For the Mth generator among the N+1 generators connected in series, an image obtained by upsampling the generated image output by the M-1th generator and the first test paper image that has been downsampled N-M+1 times are input into the Mth generator to obtain a generated image corresponding to the Mth generator, where M is greater than or equal to 2 and less than or equal to N+1; The generated image output by the Nth generator is used as the target test paper image.

5. The method according to claim 3, characterized in that The training of the test paper image generation model includes: Get sample test images of the target style transfer type; Inputting the sample test paper image into an initial test paper image generation model, wherein the initial test paper image generation model comprises N+1 untrained generators connected in series, wherein each untrained generator is connected to a discriminator; For the first generator among the N+1 untrained generators connected in series, random noise is input into the first generator to obtain a generated image corresponding to the first generator; the generated image corresponding to the first generator and a sample test paper image that has been downsampled N times are input into the discriminator corresponding to the first generator to obtain a discrimination result output by the discriminator corresponding to the first generator, and the parameters of the first generator are adjusted according to the discrimination result to obtain a trained first generator; For the Mth generator among the N+1 untrained generators connected in series, the random noise that has been upsampled M-1 times and the generated image output by the trained M-1th generator are input into the Mth generator to obtain the generated image corresponding to the Mth generator; the generated image corresponding to the Mth generator and the sample test paper image that has been downsampled N-M+1 times are input into the discriminator corresponding to the Mth generator to obtain the discrimination result output by the discriminator corresponding to the Mth generator, and the parameters of the Mth generator are adjusted according to the discrimination result to obtain the trained Mth generator, where M is greater than or equal to 2 and less than or equal to N+1; According to the trained 1st generator to the N+1th generator, a trained test paper image generation model corresponding to the target style transfer type is obtained.

6. The method according to claim 5, characterized in that The generator and the discriminator are both composed of five convolutional blocks, each convolutional block includes a convolutional layer, a batch normalization layer and an activation layer, and the generator is composed of a fully convolutional network.

7. A test paper image generating device, characterized in that: The device comprises: a first generating module configured to generate a test paper layout based on a pre-acquired material library, and to add test questions based on the test paper layout to obtain a first test paper image including a plurality of annotation information, wherein each material in the material library includes at least one type of annotation information; a second generation module, configured to input the first test paper image into any one of a plurality of pre-trained test paper image generation models, to obtain a target test paper image having multiple annotated information after style transfer, so that the target test paper image appears to be an image actually taken by a user, wherein each of the test paper image generation models has a different style transfer type; The device further comprises: A first acquisition module is used to obtain a public dataset with annotations as a training set, pre-train the initial semantic segmentation model, and obtain a trained first semantic segmentation model; a second acquisition module for acquiring a set of test paper images without annotated information, and annotating a target number of test papers in the test paper image set based on the annotation categories of the public dataset, thereby obtaining a first annotated image set and a second image set without annotated information, such that the annotation categories of the first annotated image set are consistent with the annotation categories of the public dataset, and the number of test paper images in the first image set is smaller than that in the second image set; a transfer learning module, configured to input the first image set into the first semantic segmentation model, perform transfer learning, and obtain a second semantic segmentation model; The semantic segmentation module is used to input the second image set into the second semantic segmentation model, perform semantic segmentation on the second image set, and obtain the material library, wherein each material in the material library includes annotation information corresponding to the material.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 6.

Images