Sample image generation method, apparatus and device

Abstract

The present disclosure provides a sample image generation method, device and equipment, relates to the technical field of artificial intelligence, in particular to the technical field of computer vision, image processing, deep learning and the like. The specific implementation scheme is: after determining a plurality of checkbox regions in an original image, at least one first target checkbox region is selected from the plurality of checkbox regions, a first target checkbox image matching the gray mean value of the first target checkbox region is obtained from a first checkbox image set, the first target checkbox region in the original image is replaced with the first target checkbox image to obtain a target image, and a sample image set for training a checkbox detection model is generated according to the original image and the target image. Thus, a way of generating a sample image for training a checkbox detection model by processing an original image is provided, and the cost of obtaining a sample image for training a checkbox detection model is reduced.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, specifically to technical fields such as computer vision, image processing, and deep learning, and in particular to methods, apparatus, and devices for generating sample images. Background Technology

[0002] In some scenarios, it is necessary to detect checkboxes in image data. In related technologies, in order to detect image data containing checkboxes using a checkbox detection model, it is usually necessary to train the checkbox detection model. However, during the training process, sample images containing checkboxes are difficult to obtain, and the training cost of the checkbox detection model is high. Summary of the Invention

[0003] This disclosure provides a method, apparatus, and device for generating sample images.

[0004] According to one aspect of this disclosure, a method for generating a sample image is provided, comprising: determining a plurality of checkbox regions in an original image; selecting at least one first target checkbox region from the plurality of checkbox regions; obtaining a first target checkbox image from a preset first checkbox image set whose grayscale mean matches that of the first target checkbox region, wherein the checkbox images in the first checkbox image set are images that only include checkboxes in a selected state and are labeled with a first label, wherein the first label indicates that the checkbox images in the first checkbox image set are in a selected state; replacing the first target checkbox region in the original image with the first target checkbox image to obtain a target image; and generating a sample image set for training a checkbox detection model based on the original image and the target image.

[0005] According to another aspect of this disclosure, a sample image generation apparatus is provided, comprising: a first determining module, configured to determine a plurality of checkbox regions in an original image; a selecting module, configured to select at least one first target checkbox region from the plurality of checkbox regions; a first acquiring module, configured to acquire a first target checkbox image from a preset first checkbox image set whose grayscale mean matches that of the first target checkbox region, wherein the checkbox images in the first checkbox image set are images that only include checkboxes in a selected state and are labeled with a first label, wherein the first label indicates that the checkbox images in the first checkbox image set are in a selected state; a first replacing module, configured to replace the first target checkbox region in the original image with the first target checkbox image to obtain a target image; and a generating module, configured to generate a sample image set for training a checkbox detection model based on the original image and the target image.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a sample image generation method of this disclosure.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform a method for generating sample images disclosed in embodiments of this disclosure.

[0008] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method for generating sample images of this disclosure.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0011] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0012] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0013] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0014] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0015] Figure 5 This is a schematic diagram according to the fifth embodiment of the present disclosure;

[0016] Figure 6 Examples of the original image and the target image obtained by processing the original image. Figure 1 ;

[0017] Figure 7 Examples of the original image and the target image obtained by processing the original image. Figure 2 ;

[0018] Figure 8 This is a schematic diagram according to the sixth embodiment of the present disclosure;

[0019] Figure 9 This is a schematic diagram according to the seventh embodiment of the present disclosure;

[0020] Figure 10 This is a block diagram of an electronic device used to implement the sample image generation method of the embodiments of this disclosure. Detailed Implementation

[0021] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0022] The method, apparatus, and device for generating sample images according to embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0023] Figure 1 This is a schematic diagram according to the first embodiment of the present disclosure.

[0024] like Figure 1 As shown, the method for generating this sample image may include:

[0025] Step 101: Identify multiple checkbox areas in the original image.

[0026] It should be noted that the entity executing the above-mentioned sample image generation method is a sample image generation device, which can be implemented by software and / or hardware. In this embodiment, the sample image generation device can be an electronic device, or it can be configured in an electronic device.

[0027] In this example embodiment, the electronic device may include, but is not limited to, terminal devices and servers, etc. This embodiment does not limit the electronic device.

[0028] In some exemplary embodiments, one possible way to determine multiple checkbox regions in the original image is to: obtain the checkbox annotation results of the original image; and determine multiple checkbox regions in the original image based on the checkbox annotation results. Thus, multiple checkbox regions in the original image are accurately determined using the checkbox annotation results of the original image.

[0029] The checkbox region refers to the image region that includes the checkboxes. For example, the checkbox region could be the image region in the original image containing the bounding rectangle of the complete checkbox.

[0030] It is understandable that the checkbox annotation results are used to annotate the positions of checkboxes in the original image.

[0031] As an exemplary implementation, checkboxes in the original image can be manually annotated using rectangles of a specified color to obtain the checkbox annotation results for the original image. For example, the specified color can be green, meaning that green rectangles can be used to manually annotate the positions of each checkbox in the original image to obtain the checkbox annotation results for the original image.

[0032] It is understood that in practical applications, other methods can be used to annotate the checkboxes in the original image. This embodiment does not specifically limit the method of annotating the checkboxes in the original image.

[0033] In some other exemplary embodiments, a checkbox detection algorithm can also be used to detect checkboxes in the original image to obtain checkbox detection results for the original image, wherein the checkbox detection results may include each checkbox region in the original image.

[0034] In some examples, the checkbox detection results may also include the location information of each checkbox area.

[0035] The location information is used to indicate the position of the corresponding checkbox area in the original image.

[0036] The original image can be obtained by processing a document with checkboxes. For example, the original image can be obtained by capturing or scanning a paper document with checkboxes.

[0037] In this example, there can be multiple original images; for example, there can be 500 original images.

[0038] It is understood that each original image can be processed in the manner disclosed in this example to obtain at least one target image corresponding to each original image.

[0039] Step 102: Select at least one first target check box area from the multiple check box areas.

[0040] In some exemplary embodiments, the number of checkboxes in the selected state among multiple checkboxes can be determined, and L checkboxes are randomly selected from the multiple checkboxes based on the number L of the selected checkboxes, and the selected checkboxes are referred to as the first target checkboxes.

[0041] As an example, if the number L of selected checkboxes is equal to 1, it means that the selection type of the checkboxes in the original image is single selection.

[0042] As another example, if the number L of selected checkboxes is greater than 2, it indicates that the selection type of the checkboxes in the original image is multiple selection.

[0043] In some examples, one possible way to determine the number of selected checkboxes in multiple checkbox regions is to obtain the selection status annotation result corresponding to the original image, and determine the number of selected checkboxes in multiple checkbox regions based on the selection status annotation result, wherein the selection status annotation result is obtained by annotating the number of selected checkboxes in the original image.

[0044] It is understandable that the selection status annotation of the original image will be different depending on the number of checkboxes that are selected in the original image.

[0045] In this embodiment, the annotation method for the selected state annotation result is not specifically limited. In practical applications, the annotation method for the selected state annotation result can be set according to actual needs.

[0046] Step 103: Obtain the first target checkbox image from the preset first checkbox image set that matches the grayscale mean of the first target checkbox region.

[0047] The first set of checkbox images consists of images of checkboxes that are selected, and each checkbox image in the first set of checkbox images is labeled with a first label, which indicates that the checkbox image in the first set of checkbox images is selected.

[0048] In some examples, the matching degree of grayscale mean between the first target checkbox region and each checkbox image in the first checkbox image set can be determined, and checkbox images with a matching degree greater than or equal to a preset matching degree threshold can be selected from the first checkbox image set, and the selected checkbox images can be used as the first target checkbox images.

[0049] In some exemplary embodiments, the shape of the checkbox in the first target checkbox area in this example can be the same as the shape of the checkbox in each checkbox image in the first checkbox image set. For example, the shape of the checkbox can be a rectangle.

[0050] In some other exemplary embodiments, in order to ensure that the shapes of the checkboxes in the subsequently generated target image and the original image are the same, when the shapes of the checkboxes in the first set of checkbox images include multiple types, in some examples, the shape of the checkbox in the first target checkbox area can be determined, and multiple candidate checkbox images with the same shape as the checkboxes in the first checkbox area can be obtained from the preset first set of checkbox images, and the first target checkbox image that matches the grayscale mean of the first target checkbox area can be selected from the multiple candidate checkbox images.

[0051] In this example, the number of first target checkbox images can be one or more.

[0052] Step 104: Replace the first target checkbox area in the original image with the first target checkbox image to obtain the target image.

[0053] In some examples, the first target checkbox area in the original image can be replaced with the first target checkbox image based on the location information of the first target checkbox area to obtain the target image.

[0054] In some exemplary embodiments, when there are multiple first target checkbox images, multiple first target checkbox images can be used to replace the first target checkbox area in the original image to obtain multiple target images.

[0055] Step 105: Generate a set of sample images for training the checkbox detection model based on the original image and the target image.

[0056] In some exemplary implementations, the original image and the target image can be merged to obtain a merged image set, and the images in the merged image set can be labeled to obtain a sample image set that can be used to train a checkbox detection model.

[0057] The sample image generation method provided in this disclosure, after determining multiple checkbox regions in the original image, selects at least one first target checkbox region from the multiple checkbox regions, obtains a first target checkbox image from a preset first checkbox image set whose grayscale mean matches that of the first target checkbox region, and replaces the first target checkbox region in the original image with the first target checkbox image to obtain a target image. Furthermore, it generates a sample image set for training a checkbox detection model based on the original image and the target image. Therefore, it provides a method for generating sample images for training a checkbox detection model by processing the original image, reducing the cost of obtaining sample images for training the checkbox detection model, and thus reducing the training cost of the checkbox detection model.

[0058] Based on the above embodiments, in some scenarios, the sizes of the first target checkbox image and the first target checkbox region may differ. To further improve the quality of the generated target image, in some exemplary embodiments, before replacing the first target checkbox region in the original image with the first target checkbox image to obtain the target image, the first target checkbox image can be scaled according to the size of the first target checkbox region, wherein the size of the scaled first target checkbox image is the same as the size of the first target checkbox region. Then, the first target checkbox image in the original image is replaced with the scaled first target checkbox image to obtain the target image.

[0059] In some exemplary embodiments, the first target checkbox image in the original image can be replaced with a scaled first target checkbox image based on the position information of the first target checkbox image to obtain the target image.

[0060] In some examples, the first target checkbox image can be scaled using bilinear interpolation so that the size of the scaled first target checkbox image is the same as the size of the first target checkbox area.

[0061] In some exemplary embodiments, in order to clearly understand how to obtain the first target checkbox image that matches the grayscale mean of the first target checkbox region from a preset first checkbox image set, the following describes the process in conjunction with... Figure 2 The process is described exemplarily.

[0062] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure.

[0063] like Figure 2 As shown, the method for generating this sample image may include:

[0064] Step 201: Identify multiple checkbox areas in the original image.

[0065] Step 202: Select at least one first target check box area from the multiple check box areas.

[0066] It should be noted that the specific implementation methods of steps 201 to 202 can be referred to the relevant descriptions of the embodiments of this disclosure, and will not be repeated here.

[0067] Step 203: Determine the absolute value of the difference between the mean gray value of the first target checkbox region and the mean gray value of each checkbox image in the first checkbox image set.

[0068] The first set of checkbox images consists of images of checkboxes that are selected, and each checkbox image is labeled with a first label, which indicates that the checkbox images in the first set of checkbox images are selected.

[0069] In this context, a checkbox image being selected means that the interior of the checkbox in the checkbox image meets a preset condition. For example, the preset condition may be that the interior of the checkbox includes a specified symbol, such as a checkmark, cross, or dot. Alternatively, the preset condition may be that the interior of the checkbox is painted with a specified color, such as black or blue. This embodiment does not specifically limit this.

[0070] Step 204: Sort the checkbox images in the first checkbox image set according to the absolute value of the difference from smallest to largest to obtain the sorting result.

[0071] Step 205: Obtain the checkbox image that ranks in the top K positions from the sorting results as the first target checkbox image, where K is a positive integer and is less than the total number of checkbox images in the first checkbox image set.

[0072] Wherein, K is a preset value, for example, K can be 5, or 20, etc. In practical applications, the value of K can be set according to actual needs, and this embodiment does not make specific limitations in this regard.

[0073] Step 206: Replace the first target checkbox area in the original image with the first target checkbox image to obtain the target image.

[0074] Step 207: Generate a set of sample images for training the checkbox detection model based on the original image and the target image.

[0075] It should be noted that the specific implementation methods of steps 206 and 207 can be found in the relevant descriptions of the embodiments of this disclosure, and will not be repeated here.

[0076] In this example, the absolute value of the difference between the grayscale mean of the first target checkbox region and each checkbox image in the first checkbox image set is determined. The checkbox images in the first checkbox image set are then sorted in ascending order of the absolute value of the difference to obtain a sorting result. The checkbox image ranked in the top K positions of the sorting result is selected as the first target checkbox image corresponding to the first target checkbox region. Thus, the first target checkbox image with a small grayscale difference from the first checkbox region is quickly and accurately obtained from the first checkbox image set, thereby improving the quality of the subsequently generated target image.

[0077] To facilitate a clear understanding of this disclosure, the following will be combined with... Figure 3 The method for generating sample images in this embodiment will be further described exemplarily.

[0078] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure.

[0079] like Figure 3 As shown, the method for generating this sample image may include:

[0080] Step 301: Identify multiple checkbox areas in the original image.

[0081] It should be noted that the specific method of step 301 can be found in the relevant description of the embodiments of this disclosure, and will not be repeated here.

[0082] Step 302: Obtain the text content annotation results of the original image.

[0083] Step 303: Based on the text content annotation results, determine multiple text content regions in the original image.

[0084] Step 304: Determine the distance between each checkbox area and each text content area.

[0085] In some examples, the distance between each checkbox area and each text content area can be determined based on the position information of each checkbox area and each text content area.

[0086] Step 305: For each checkbox area, select the text area that is closest to the checkbox area from the multiple text content areas as the target text content area corresponding to the checkbox area.

[0087] Step 306: Perform character recognition on the target text content area and use the character recognition results to annotate the target text content area.

[0088] In some examples, optical character recognition (OCR) technology can be used to recognize characters in the target text content region to obtain the character recognition result, and then the target text content region in the original image can be labeled using the character recognition result.

[0089] Step 307: Select at least one first target check box area from the multiple check box areas.

[0090] Step 308: Obtain the first target checkbox image from the preset first checkbox image set that matches the grayscale mean of the first target checkbox region.

[0091] The first set of checkbox images consists of images of checkboxes that are selected, and each checkbox image in the first set of checkbox images is labeled with a first label.

[0092] The first label indicates that the checkbox image in the first set of checkbox images is selected.

[0093] Step 309: Replace the first target checkbox area in the original image with the first target checkbox image to obtain the target image.

[0094] Step 310: Merge the original image and the target image to obtain a set of sample images for training the checkbox detection model.

[0095] It should be noted that the specific implementation methods of steps 307 to 310 can be found in the relevant descriptions of the embodiments of this disclosure, and will not be repeated here.

[0096] In this example, by annotating the text content regions corresponding to each checkbox region in the original image, the target image obtained based on the original image also has annotation results. This eliminates the need to annotate the target image, reducing the cost of annotating the target image. This further reduces the cost of obtaining sample images for training the checkbox detection model and improves the efficiency of obtaining sample images for training the checkbox detection model.

[0097] To facilitate a clear understanding of this disclosure, the following will be combined with... Figure 4 The method for generating sample images in this embodiment will be further described exemplarily.

[0098] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure.

[0099] like Figure 4 As shown, the method for generating this sample image may include:

[0100] Step 401: Identify multiple checkbox areas in the original image.

[0101] Step 402: Select at least one first target check box area from the multiple check box areas.

[0102] Step 403: Obtain the first target checkbox image from the preset first checkbox image set that matches the grayscale mean of the first target checkbox region.

[0103] The first set of checkbox images consists of images of checkboxes that are selected, and each checkbox image in the first set of checkbox images is labeled with a first label.

[0104] The first label indicates that the checkbox image in the first set of checkbox images is selected.

[0105] It should be noted that the specific implementation methods of steps 401 to 403 can be found in the relevant descriptions of the embodiments of this disclosure, and will not be repeated here.

[0106] Step 404: Determine that a second target checkbox area exists in the checkbox areas other than the first target checkbox area among the multiple checkbox areas, wherein the checkboxes in the second target checkbox area are in the selected state.

[0107] In some exemplary implementations, multiple checkbox areas other than the first target checkbox area can be obtained, and it can be determined whether there are checkbox areas in the selected state among the other checkbox areas. If so, the checkbox areas in the selected state among the other checkboxes are referred to as the second target checkbox area.

[0108] Step 405: Obtain a second target checkbox image from the preset second checkbox image set that matches the grayscale mean of the second target checkbox region.

[0109] The second set of checkbox images consists of images of checkboxes that are not selected, and the checkbox images in the second set of checkbox images are labeled with a second label.

[0110] The second label indicates that the checkbox images in the second set of checkbox images are in an unselected state.

[0111] Step 406: Replace the second target checkbox area in the original image with the second target checkbox image.

[0112] Step 407: Replace the first target checkbox area in the original image with the first target checkbox image to obtain the target image.

[0113] Step 408: Generate a set of sample images for training the checkbox detection model based on the original image and the target image.

[0114] It should be noted that the specific implementation methods of steps 407 and 408 can be found in the relevant descriptions in the embodiments of this disclosure, and will not be repeated here.

[0115] In this example, multiple checkbox areas other than the first target checkbox area are obtained. If a second target checkbox area exists in the other checkbox areas and is in a selected state, the second target checkbox area is processed using the set of second checkbox images that are not in a selected state. This ensures that the number of selected checkbox areas in the subsequently generated target image is the same as the number of selected checkbox areas in the original image.

[0116] To facilitate a clear understanding of this disclosure, the following will be combined with... Figure 5 The method for generating sample images in this embodiment is described by way of example.

[0117] Figure 5 This is a schematic diagram according to the fifth embodiment of the present disclosure.

[0118] like Figure 5 As shown, the method for generating this sample image may include:

[0119] Step 501: Obtain multiple raw images.

[0120] Step 502: Perform grayscale processing on multiple original images respectively.

[0121] In some exemplary embodiments, during the process of checking box detection in an image using a checking box detection model, to improve the processing speed and efficiency of the model, the grayscale image to be processed is typically used. This grayscale image is obtained by converting the original image to grayscale. Therefore, in training the checking box detection model, related technologies usually train the initial model based on grayscale sample images. To conveniently obtain the grayscale sample image from the original image, in some examples, the original image can be grayscale processed first.

[0122] Step 503: For each original image, key-value pairs can be used to annotate each checkbox area and the corresponding text content area on the original image.

[0123] In this key-value pair, the key is the text content area in the original image, and the corresponding value is the checkbox area corresponding to that text content area.

[0124] Step 504: Calculate the average gray value of each checkbox region labeled with a value in the original image to obtain the average gray value of each checkbox region labeled with a value.

[0125] Step 505: Perform text recognition on the text content regions corresponding to each checkbox area in the original image to obtain the text content corresponding to each text content region, and use the text content to annotate the corresponding text content regions.

[0126] Step 506: If it is determined that there is one checkbox area in the original image that is selected, a first target checkbox area is randomly selected from the multiple checkbox areas in the original image.

[0127] Step 507: Obtain the first target checkbox image from the preset first checkbox image set that matches the grayscale mean of the first target checkbox region.

[0128] The first set of checkbox images consists of images of checkboxes that are selected, and each checkbox image is labeled with a first label, which indicates that the checkbox images in the first set of checkbox images are selected.

[0129] In this example, the shape of the checkbox in the first target checkbox area is the same as the shape of the checkbox in each checkbox image in the first checkbox image set, which is used as an example for illustrative description.

[0130] Step 508: In determining multiple checkbox areas, there exists a second target checkbox area in the checkbox areas other than the first target checkbox area, wherein the checkboxes in the second target checkbox area are in the selected state.

[0131] It is understood that if it is determined that there are no selected checkboxes in the multiple checkbox areas other than the first target checkbox area, step 511 can be executed directly.

[0132] Step 509: Obtain a second target checkbox image from the preset second checkbox image set that matches the grayscale mean of the second target checkbox region.

[0133] The second set of checkbox images consists of images of checkboxes that are not selected, and the checkbox images in the second set of checkbox images are labeled with a second label.

[0134] The second label indicates that the checkbox images in the second set of checkbox images are in an unselected state.

[0135] In this example, the shape of the checkbox in the second target checkbox area is the same as the shape of the checkbox in each checkbox image in the first checkbox image set, which is used as an example for illustrative description.

[0136] It should be noted that in this example, the first set of checkbox images and the second set of checkbox images can be obtained by converting the images in the collected set of images containing only checkboxes to grayscale and then labeling the grayscale image set with its type.

[0137] Step 510: Replace the second target checkbox area in the original image with the second target checkbox image.

[0138] Step 511: Replace the first target checkbox area in the original image with the first target checkbox image to obtain the target image.

[0139] For example, the original image, such as Figure 6 As shown in Figure a, if it is determined that there is only one selected checkbox area in the original image, and if it is desired that the target image generated based on the original image also has only one selected checkbox area, then a first target checkbox area can be randomly selected from the multiple checkbox areas in the original image. For Figure 6 In image a, there are 7 checkbox areas, labeled with numbers 1, 2, 3...7. Correspondingly, after selecting the first target checkbox area from image a as the checkbox area corresponding to number 4, a first target checkbox image matching the average grayscale value of this first target checkbox area can be obtained from the set of first checkbox images in the selected state. Then, it is determined whether any of the other 7 checkbox areas besides the first target checkbox area are in a selected state. It is found that the checkbox area corresponding to number 6 in the other checkbox areas is in a selected state. At this point, a second target checkbox image matching the average grayscale value of the checkbox area corresponding to number 6 can be obtained from the set of second checkbox images in the unselected state. The checkbox area corresponding to number 6 in the original image is replaced with the second target checkbox image, and the first target checkbox area in the original image is also replaced with the first target checkbox image to obtain the target image. The target image, as shown... Figure 6 As shown in Figure b.

[0140] Step 512: If it is determined that there are L checkbox areas in the original image that are selected, randomly select N first target checkbox areas from the multiple checkbox areas in the original image.

[0141] Where L is an integer greater than 1.

[0142] Step 513: For each first target checkbox area, obtain the first target checkbox image that matches the grayscale mean of the first target checkbox area from the preset first checkbox image set.

[0143] The first set of checkbox images consists of images of checkboxes that are selected, and each checkbox image in the first set of checkbox images is labeled with a first label.

[0144] The first label indicates that the checkbox image in the first set of checkbox images is selected.

[0145] In this example, the shape of the checkbox in the first target checkbox area is the same as the shape of the checkbox in each checkbox image in the first checkbox image set, which is used as an example for illustrative description.

[0146] Step 514: In determining that there is a second target checkbox area in the multiple checkbox areas of the original image other than the first target checkbox area, the checkboxes in the second target checkbox area are in the selected state.

[0147] It is understood that if it is determined that there are no selected checkboxes in the multiple checkbox areas of the original image other than the first target checkbox area, then step 517 can be executed directly.

[0148] Step 515: Obtain a second target checkbox image from the preset second checkbox image set that matches the grayscale mean of the second target checkbox region.

[0149] The second set of checkbox images consists of images of checkboxes that are not selected, and is labeled with a second label indicating that the checkbox images in the second set of checkbox images are not selected.

[0150] Step 516: Replace the second target checkbox area in the original image with the second target checkbox image.

[0151] Step 517: Replace the L first target checkbox regions in the original image with the first target checkbox images corresponding to the L second target checkbox regions to obtain the target image.

[0152] For example, the original image, such as Figure 7 As shown in Figure a, if we determine that there are three selected checkbox areas in the original image, and we want the target image generated from the original image to have three selected checkbox areas, then we can randomly select three first target checkbox areas from the multiple checkbox areas in the original image. Specifically, for... Figure 7In image a, there are 5 checkbox areas, labeled with numbers 1, 2, 3...5. Assuming the three selected first target checkbox areas are those corresponding to numbers 3, 4, and 5, for each first target checkbox area, an image matching the average grayscale value of that first target checkbox area can be obtained from the set of selected first checkbox images. Figure 7 In Figure a, the five checkbox areas, excluding the first target checkbox area, are the checkbox areas corresponding to the numbers 1 and 2. It can be determined that both the checkbox areas corresponding to the numbers 1 and 2 are selected. Therefore, a second target checkbox image matching the grayscale mean of the checkbox area corresponding to the number 1 can be obtained from the set of second checkbox images in the non-selected state, and the checkbox area corresponding to the number 1 in the original image can be replaced with the second target checkbox image. Additionally, a third target checkbox image matching the grayscale mean of the checkbox area corresponding to the number 2 can be obtained from the set of second checkbox images in the non-selected state, and the checkbox area corresponding to the number 2 in the original image can be replaced with the third target checkbox image. Finally, the first target checkbox area in the original image can be replaced with the first target checkbox image to obtain the target image. The target image, as shown... Figure 7 As shown in Figure b.

[0153] In this example, by combining checkbox images from the checkbox image set and processing the checkbox regions in the original image, high-quality sample images needed to train the checkbox detection model can be generated. This reduces the cost of obtaining sample images for training the checkbox detection model, thereby reducing the training cost of the subsequent training of the checkbox detection model.

[0154] To implement the above embodiments, this disclosure also provides a sample image generation apparatus.

[0155] Figure 8 This is a schematic diagram according to the sixth embodiment of the present disclosure.

[0156] like Figure 8 As shown, the sample image generation device 800 may include: a first determining module 801, a selecting module 802, a first acquiring module 803, a first replacing module 804, and a generating module 805, wherein:

[0157] The first determining module 801 is used to determine multiple checkbox areas in the original image.

[0158] Selection module 802 is used to select at least one first target check box area from a plurality of check box areas.

[0159] The first acquisition module 803 is used to acquire a first target checkbox image that matches the grayscale mean of the first target checkbox region from a preset first checkbox image set. The checkbox images in the first checkbox image set are images that only include checkboxes in the selected state and are labeled with a first label, wherein the first label indicates that the checkbox images in the first checkbox image set are in the selected state.

[0160] The first replacement module 804 is used to replace the first target checkbox area in the original image with the first target checkbox image to obtain the target image.

[0161] The generation module 805 is used to generate a set of sample images for training the checkbox detection model based on the original image and the target image.

[0162] It should be noted that the explanation of the above-described method for generating sample images also applies to this embodiment, and will not be repeated here.

[0163] The sample image generation apparatus of this disclosure, after determining multiple checkbox regions in an original image, selects at least one first target checkbox region from the multiple checkbox regions, obtains a first target checkbox image from a preset first checkbox image set whose grayscale mean matches that of the first target checkbox region, and replaces the first target checkbox region in the original image with the first target checkbox image to obtain a target image. It also generates a sample image set for training a checkbox detection model based on the original image and the target image. Therefore, it provides a method for generating sample images for training a checkbox detection model by processing the original image, reducing the cost of obtaining sample images for training the checkbox detection model, and thus reducing the training cost of training the checkbox detection model.

[0164] In one embodiment of this disclosure, Figure 9 This is a schematic diagram based on the seventh embodiment of the present disclosure, as shown below. Figure 9 As shown, the sample image generation device 900 may include: a first determining module 901, a selection module 902, a first acquiring module 903, a first replacing module 904, a generating module 905, a scaling module 906, a grayscale processing module 907, a second acquiring module 908, a second determining module 909, a third determining module 910, a fourth determining module 911, a character recognition module 912, a fifth determining module 913, a third acquiring module 914, and a second replacing module 915.

[0165] It should be noted that for detailed descriptions of the selection module 902, the first replacement module 904, and the generation module 905, please refer to [the relevant documentation / reference]. Figure 8The descriptions of the selection module 802, the first replacement module 804, and the generation module 805 in the illustrated embodiment will not be repeated here.

[0166] In one embodiment of this disclosure, the device may further include: a scaling module 906, configured to scale the first target checkbox image according to the size of the first target checkbox area, wherein the size of the scaled first target checkbox image is the same as the size of the first target checkbox area.

[0167] In one embodiment of this disclosure, the first acquisition module 903 is specifically configured to: determine the absolute value of the difference between the grayscale mean value of the first target checkbox region and the grayscale mean value of each checkbox image in the first checkbox image set; sort the checkbox images in the first checkbox image set according to the order of the absolute value of the difference from smallest to largest to obtain a sorting result; and obtain the checkbox image ranked in the top K positions from the sorting result as the first target checkbox image, where K is a positive integer and is less than the total number of checkbox images in the first checkbox image set.

[0168] In one embodiment of this disclosure, the first determining module 901 is specifically used to: obtain the checkbox annotation results of the original image; and determine multiple checkbox regions in the original image based on the checkbox annotation results.

[0169] In one embodiment of this disclosure, the apparatus may further include:

[0170] The grayscale processing module 907 is used to perform grayscale processing on the original image.

[0171] In one embodiment of this disclosure, the apparatus may further include:

[0172] The second acquisition module 908 is used to acquire the text content annotation results of the original image;

[0173] The second determining module 909 is used to determine multiple text content regions in the original image based on the text content annotation results;

[0174] The third determining module 910 is used to determine the distance between each checkbox area and each text content area;

[0175] The fourth determining module 911 is used to select the text region closest to the check box region from multiple text content regions for each check box region as the target text content region corresponding to the check box region.

[0176] The text recognition module 912 is used to perform text recognition on the target text content area and to annotate the target text content area using the text recognition results.

[0177] In one embodiment of this disclosure, the apparatus may further include:

[0178] The fifth determining module 913 is used to determine that a second target check box area exists in multiple check box areas other than the first target check box area, wherein the check box in the second target check box area is in the selected state;

[0179] The third acquisition module 914 is used to acquire a second target checkbox image that matches the grayscale mean of the second target checkbox region from a preset second checkbox image set. The checkbox images in the second checkbox image set are images that only include checkboxes in an unselected state, and are labeled with a second label, which indicates that the checkbox images in the second checkbox image set are in an unselected state.

[0180] The second replacement module 915 is used to replace the second target checkbox area in the original image with the second target checkbox image.

[0181] It should be noted that the foregoing explanation of the sample image generation method embodiment also applies to the sample image generation device in this embodiment, and will not be repeated here.

[0182] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0183] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0184] like Figure 10 As shown, the electronic device 1000 may include a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of the device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0185] Multiple components in device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0186] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as the method for generating sample images. For example, in some embodiments, the method for generating sample images may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the method for generating sample images described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform a method for generating sample images by any other suitable means (e.g., by means of firmware).

[0187] Various embodiments of the apparatuses and techniques described above herein can be implemented in digital electronic circuit devices, integrated circuit devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), device-on-a-chip (SoC) devices, complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable device including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage device, at least one input device, and at least one output device, and transmitting data and instructions to the storage device, the at least one input device, and the at least one output device.

[0188] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0189] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution apparatus, device, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0190] To provide interaction with a user, the apparatus and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of apparatus can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0191] The apparatus and techniques described herein can be implemented in computing devices that include backend components (e.g., as a data server), or computing devices that include middleware components (e.g., an application server), or computing devices that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the apparatus and techniques described herein), or computing devices that include any combination of such backend, middleware, or frontend components. The components of the apparatus can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0192] Computer devices can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. A server can be a cloud server, a distributed server, or a server incorporating blockchain technology.

[0193] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0194] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0195] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for generating a sample image, comprising: Identify multiple checkbox regions in the original image; Select at least one first target check box area from the plurality of check box areas; From a preset set of first checkbox images, a first target checkbox image that matches the average grayscale value of the first target checkbox region is obtained. The checkbox images in the first set of first checkbox images are images that only include checkboxes in the selected state and are labeled with a first label, wherein the first label indicates that the checkbox images in the first set of first checkbox images are in the selected state. It is determined that a second target check box area exists in the other check box areas besides the first target check box area among the plurality of check box areas, wherein the check box in the second target check box area is in a selected state; From a preset set of second checkbox images, a second target checkbox image is obtained that matches the grayscale mean of the second target checkbox region. The checkbox images in the second set of images consist only of images of checkboxes that are not selected, and are labeled with a second label indicating that the checkbox images in the second set of images are in an unselected state. Replace the second target checkbox area in the original image with the second target checkbox image; Replace the first target checkbox area in the original image with the first target checkbox image to obtain the target image; Based on the original image and the target image, a set of sample images is generated for training the checkbox detection model.

2. The method according to claim 1, wherein, Before replacing the first target checkbox region in the original image with the first target checkbox image to obtain the target image, the method further includes: The first target checkbox image is scaled according to the size of the first target checkbox area, wherein the size of the scaled first target checkbox image is the same as the size of the first target checkbox area.

3. The method according to claim 1, wherein, The step of obtaining a first target checkbox image from a preset first checkbox image set that matches the average grayscale value of the first target checkbox region includes: Determine the absolute value of the difference between the mean gray value of the first target checkbox region and the mean gray value of each checkbox image in the first checkbox image set; The checkbox images in the first set of checkbox images are sorted in ascending order of the absolute value of the difference to obtain a sorting result; The first target checkbox image is obtained from the sorting result, where K is a positive integer and less than the total number of checkbox images in the first set of checkbox images.

4. The method according to claim 1, wherein, Determining multiple checkbox regions in the original image includes: Obtain the checkbox annotation results of the original image; Based on the checkbox labeling results, multiple checkbox regions in the original image are determined.

5. The method according to claim 1, wherein, Before determining the multiple checkbox regions in the original image, the method further includes: The original image is converted to grayscale.

6. The method according to claim 4, wherein, The method further includes: Obtain the text content annotation results of the original image; Based on the text content annotation results, multiple text content regions in the original image are determined; Determine the distance between each of the checkbox areas and each of the text content areas; For each checkbox area, the text area that is closest to the checkbox area is selected from the plurality of text content areas and taken as the target text content area corresponding to the checkbox area; The target text content region is subjected to character recognition, and the target text content region is labeled using the character recognition results.

7. A sample image generation apparatus, comprising: The first determining module is used to determine multiple checkbox regions in the original image; The selection module is used to select at least one first target check box area from the plurality of check box areas; The first acquisition module is used to acquire a first target checkbox image that matches the grayscale mean of the first target checkbox region from a preset first checkbox image set, wherein the checkbox images in the first checkbox image set are images that only include checkboxes in the selected state and are labeled with a first label, wherein the first label indicates that the checkbox images in the first checkbox image set are in the selected state; The fifth determining module is used to determine that there is a second target check box area in the other check box areas besides the first target check box area among the plurality of check box areas, wherein the check box in the second target check box area is in a selected state; The third acquisition module is used to acquire a second target checkbox image that matches the grayscale mean of the second target checkbox region from a preset second checkbox image set, wherein the checkbox images in the second checkbox image set are images that only include checkboxes in an unselected state, and are labeled with a second label, the second label indicating that the checkbox images in the second checkbox image set are in an unselected state; The second replacement module is used to replace the second target checkbox area in the original image with the second target checkbox image; The first replacement module is used to replace the first target checkbox area in the original image with the first target checkbox image to obtain the target image; The generation module is used to generate a set of sample images for training the checkbox detection model based on the original image and the target image.

8. The apparatus according to claim 7, wherein, The device further includes: The scaling module is used to scale the first target checkbox image according to the size of the first target checkbox area, wherein the size of the scaled first target checkbox image is the same as the size of the first target checkbox area.

9. The apparatus according to claim 7, wherein, The first acquisition module is specifically used for: Determine the absolute value of the difference between the mean gray value of the first target checkbox region and the mean gray value of each checkbox image in the first checkbox image set; The checkbox images in the first set of checkbox images are sorted in ascending order of the absolute value of the difference to obtain a sorting result; The first target checkbox image is obtained from the sorting result, where K is a positive integer and less than the total number of checkbox images in the first set of checkbox images.

10. The apparatus according to claim 7, wherein, The first determining module is specifically used for: Obtain the checkbox annotation results of the original image; Based on the checkbox labeling results, multiple checkbox regions in the original image are determined.

11. The apparatus according to claim 7, wherein, The device further includes: The grayscale processing module is used to perform grayscale processing on the original image.

12. The apparatus according to claim 10, wherein, The device further includes: The second acquisition module is used to acquire the text content annotation results of the original image; The second determining module is used to determine multiple text content regions in the original image based on the text content annotation results; The third determining module is used to determine the distance between each of the checkbox areas and each of the text content areas; The fourth determining module is used to, for each of the checkbox areas, select the text area closest to the checkbox area from the plurality of text content areas as the target text content area corresponding to the checkbox area; The text recognition module is used to perform text recognition on the target text content area and to annotate the target text content area using the text recognition results.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-6.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-6.

Images