Test text picture generation method and device and label quality determination method

Abstract

The present specification provides a test text picture generation method and device and a label quality determination method. Specifically, the target text picture to be labeled can be preprocessed first to obtain text character data and blank area data in the target text picture. Then, according to the text character data and the blank area data, corresponding buried point text is set in the target text picture. Thus, a test text picture with buried point text having a high matching degree with the original text character and good test effect can be automatically generated. Subsequently, the test text picture can be used to determine the label quality of the target object responsible for labeling through corresponding tests.

Description

Technical Field

[0001] This manual belongs to the field of Internet technology, and in particular relates to methods and devices for generating test text images and methods for determining annotation quality. Background Technology

[0002] When training an OCR (Optical Character Recognition) model, it is often necessary for annotators to first label sample images, obtaining labeled sample images. These labeled sample images are then used to train the model. Clearly, the quality of the annotators' work affects the data quality of the labeled sample images, which in turn impacts the training of the OCR recognition model.

[0003] Currently, there is an urgent need for a method that can accurately determine the quality of annotation by annotators. Summary of the Invention

[0004] This specification provides a method, apparatus, and method for determining the quality of test text images. It can automatically generate test text images with embedded text that has a high degree of matching with the original text characters and has good test results. Then, the annotation quality of the target object to be annotated can be determined more accurately using the test text images.

[0005] The method, apparatus, and method for determining the annotation quality of the test text images provided in this manual are implemented as follows:

[0006] A method for generating a test text image includes: acquiring a target text image; wherein the target text image contains at least text characters to be labeled; preprocessing the target text image to obtain text character data and blank area data in the target text image; and setting corresponding embedded text in the target text image based on the text character data and blank area data to obtain the test text image.

[0007] A method for determining annotation quality includes: generating a corresponding test text image based on a target text image; wherein the test text image contains embedded text; the embedded text is generated based on text character data in the target text image; and using the test text image to determine the annotation quality of the target object.

[0008] A test text image generation device includes: an acquisition module for acquiring a target text image, wherein the target text image contains at least text characters to be labeled; a preprocessing module for preprocessing the target text image to acquire text character data and blank area data in the target text image; and a setting module for setting corresponding embedded text in the target text image according to the text character data and blank area data to obtain the test text image.

[0009] An apparatus for determining annotation quality includes: a generation module for generating a corresponding test text image based on a target text image; wherein the test text image contains embedded text; the embedded text is generated based on text character data in the target text image; and a determination module for determining the annotation quality of a target object using the test text image.

[0010] A server includes a processor and a memory for storing processor-executable instructions, wherein the processor executes the instructions to implement steps related to the method for generating a test text image or the method for determining annotation quality.

[0011] A computer-readable storage medium storing computer instructions thereon, which, when executed, implement the steps of the method for generating the test text image or the method for determining the annotation quality.

[0012] The method, apparatus, and method for determining annotation quality of the test text image provided in this manual can first preprocess the target text image to be annotated to obtain the text character data and blank area data in the target text image; then, based on the text character data and blank area data, corresponding embedded text can be set in the target text image, thereby efficiently and automatically generating test text images with embedded text that has a high degree of matching with the original text characters and good test results. Subsequently, the annotation quality of the target object to be annotated can be determined more accurately through corresponding tests using the aforementioned test text images. Attached Figure Description

[0013] To more clearly illustrate the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of one embodiment of the system composition of the method for determining annotation quality provided in the embodiments of this specification;

[0015] Figure 2This is a schematic diagram illustrating one embodiment of the annotation quality determination method provided in the embodiments of this specification, applied in a scenario example.

[0016] Figure 3 This is a schematic diagram illustrating one embodiment of the annotation quality determination method provided in the embodiments of this specification, applied in a scenario example.

[0017] Figure 4 This is a schematic diagram illustrating one embodiment of the annotation quality determination method provided in the embodiments of this specification, applied in a scenario example.

[0018] Figure 5 This is a schematic diagram illustrating one embodiment of the annotation quality determination method provided in the embodiments of this specification, applied in a scenario example.

[0019] Figure 6 This is a flowchart illustrating a method for generating test text images according to an embodiment of this specification;

[0020] Figure 7 This is a flowchart illustrating a method for determining annotation quality provided in one embodiment of this specification;

[0021] Figure 8 This is a schematic diagram of the structural composition of a server provided in one embodiment of this specification;

[0022] Figure 9 This is a schematic diagram of the structural composition of a test text image generation device provided in one embodiment of this specification;

[0023] Figure 10 This is a schematic diagram of the structural composition of a device for determining the quality of annotation provided in one embodiment of this specification. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0025] The method for determining annotation quality provided in the embodiments of this specification can be specifically applied to systems containing servers and clients. For details, please refer to... Figure 1 As shown. The server and client can connect via wired or wireless means to exchange data.

[0026] In this embodiment, the server may specifically include a server applied to the business platform side, capable of data transmission, data processing, and other functions, responsible for evaluating and determining the annotation quality of the annotators. Specifically, the server may be, for example, an electronic device with data processing, storage, and network interaction capabilities. Alternatively, the server may be a software program running on the electronic device, providing support for data processing, storage, and network interaction. In this embodiment, the number of servers is not specifically limited. The server may be a single server, several servers, or a server cluster formed by several servers.

[0027] In this embodiment, the client may specifically include a front-end electronic device applied to the annotator's side, capable of data acquisition, data transmission, and other functions. Specifically, the client may be, for example, a desktop computer, tablet computer, laptop computer, or smartphone. Alternatively, the client may be a software application running on the aforementioned electronic device. For example, it could be an annotation port interface of a business platform running on a desktop computer.

[0028] Typically, the server distributes the text image to be annotated to the annotator via the client. After receiving the text image, the annotator first identifies the text characters contained within it, then annotates the specific text characters as text data, presenting the annotation result as the annotation result. This result is then fed back to the server via the client, thus completing the annotation of the text image.

[0029] The current business platform wants to determine the annotation quality of the annotator in a quantitative way.

[0030] In practice, the server can first obtain multiple text images containing text characters to be labeled (which can be referred to as target text images); by processing the above multiple target text images accordingly, multiple test text images with embedded text and good test results can be obtained.

[0031] The following example illustrates the data processing steps for generating a test text image, using any one of multiple target text images (denoted as image p) as an example. The target text image p can be found in [reference needed]. Figure 2 As shown. The text characters contained in the target text image p are text characters in the form of image data.

[0032] Specifically, firstly, the server can call an existing preset OCR recognition model (which may be a low-precision OCR recognition model) to perform OCR recognition on the target text image in order to obtain text character data in the target text image that allows for errors.

[0033] Herein, the above text character data may be expressed as wp={w1, w2, ..., wi, ..., wn}. wi in w may represent the text character numbered i in the text string recognized from the target text image. For example, refer to Figure 2 as shown, the text character w3 in wp is "集", and the text character w4 is "团".

[0034] Meanwhile, an image detection model is invoked to perform blank image detection on the target text image, so as to determine blank regions that do not contain text characters in the target text image, and acquire the position data of the above blank regions in the target text image as the blank region data in the target text image.

[0035] Herein, the above blank region data may be expressed as bp={[x11,y11,x12,y12],[x21,y21,x22,y22],...,[xi1,yi1,xi2,yi2],...,[xn1,yn1,xn2,yn2]}. [xi1,yi1,xi2,yi2] in bp may specifically represent the position data of the rectangular blank region bi numbered i among the detected n rectangular blank regions. For example, refer to Figure 3 as shown, [x11,y11,x12,y12] may be the position data of the rectangular blank region b1 numbered 1.

[0036] In addition, the server may invoke a text character classification model to process the target text image, so as to perform graphic classification and recognition on the text characters in the target text image, and determine the attribute data such as font and font size of each text character in the target text image.

[0037] Herein, the attribute data of text characters may be expressed as fp={f1,f2,...,fi,..,fn}. fi in fp may specifically represent the attribute data of the text character numbered i in the text character data.

[0038] Next, the server may invoke a preset text generation model to process the text character data in the target text image, so as to generate a plurality of (usually 3) text statements matching the text characters in the target text image, which are used as buried point texts. For example, Figure 4 the buried point texts b1, b2 and b3 in the figure.

[0039] The server may first determine a plurality of blank regions in the target text image according to the blank region data; then, by detecting and according to the text paragraphs near each blank region, screen out the blank regions located at the end of a paragraph or between two adjacent paragraphs from the plurality of blank regions as the deployment positions meeting the requirements. Furthermore, the buried point text may be set at the above deployment positions. For specific reference, see Figure 4As shown, by placing embedded text at the aforementioned locations, the embedded text can be made to have a smaller difference from the surrounding original text characters and a higher degree of matching, making it appear more coordinated and uniform in the entire text image. This can reduce the errors that may occur when using the embedded text to determine the quality of the annotation later.

[0040] At the same time, the server will also record the text string of the set tracking text, as well as the location information of the placement position of the tracking text, as associated data of the tracking text.

[0041] The associated data of the embedded text in the test text image corresponding to the target text image p can be represented as DMp = {m1, m2, ..., mi ..., mn}. Specifically, mi in DMp can represent the associated data of the embedded text with ID i. Specifically, mi can be represented in the following form: mi = {si = {w1, w2, ..., wt}, bi = [xi1, yi1, xi2, yi2]}. Here, si represents the text string (or real value) of the embedded text with ID i, and bi represents the location information of the embedded text with ID i.

[0042] When setting tracking text at a specific deployment location, the server can first obtain the font size, style, and other attribute data of the existing text characters within the adjacent range of that deployment location (e.g., a range less than or equal to two characters away from the deployment location), which are Song typeface and 12pt respectively. Next, the text characters in the tracking text can also be adjusted to Song typeface and 12pt, resulting in adjusted tracking text that is less different from and more harmonious with the surrounding original text characters. Finally, the adjusted tracking text is set as image data at the deployment location, thus obtaining a test text image corresponding to the target text image.

[0043] In addition, it can also obtain and adjust the embedded text based on detailed features such as character spacing and line spacing in the original text paragraphs within the vicinity of the deployment location, so as to reduce the difference between the embedded text and the surrounding text characters.

[0044] Furthermore, after setting the embedded text at the preset location, the embedded text can be aligned with the text characters in the vicinity of the placement location. This makes the embedded text more consistent and harmonious with the surrounding text characters.

[0045] Through the above processing, see Figure 5 As shown, the test text image can achieve relatively better test results, and the set embedding text is more coordinated and consistent with the original text characters.

[0046] In the above manner, the server can process multiple target text images to obtain multiple test text images with embedded tracking text and relatively good test results; and record the association data of the tracking text in each test text image.

[0047] The server can then use the aforementioned multiple test text images to perform annotation tests on the annotators, quantitatively determining the quality of their annotations.

[0048] Specifically, the server can use the aforementioned multiple test text images to replace the target text image and send them to the client. The annotator can then receive these multiple test text images through the client.

[0049] For annotators, it is impossible to distinguish between the original text string and the embedded text in the test text image. Therefore, they will uniformly annotate the text characters found in the test text image to obtain multiple annotation results; then, the client will send the above multiple annotation results back to the server.

[0050] After receiving the annotation results, the server can evaluate only the annotation results related to the tracking text based on the associated data of the tracking text, in order to determine the annotation quality of the annotator.

[0051] Specifically, the server can determine the text string of the embedded text in each test text image and the location information of the embedded text placement position based on the correlation data of the embedded text in each test text image. Then, based on the annotation results of each test text image, the text string of the embedded text, and the location information of the embedded text placement position, the server can count the number of embedded texts annotated by the annotator in each test text image and the number of correctly annotated embedded texts. Furthermore, based on the correlation data of the embedded text in each test text image, and combining the number of embedded texts annotated by the annotator in each test text image and the number of correctly annotated embedded texts, the server can calculate evaluation parameters such as the annotation accuracy rate (Mo), the missing annotation rate (Mm), and the annotation error rate (Me) of the embedded text for that annotator. Finally, the server can determine the final annotation quality of the annotator based on the above evaluation parameters.

[0052] To utilize a single test text image (see) Figure 4 Taking the testing of annotators as an example, the server can first send the test text image to the annotator. The annotator can then retrieve the test text image through a client.

[0053] The annotator is unaware that the obtained test text image is used to test and determine the annotation quality. They will annotate the text characters found in the test text image as usual, obtain the corresponding annotation results, and then send the annotation results to the server through the client.

[0054] After receiving the annotation results from the annotator, the server can first query and retrieve the associated data of the test text image based on the image identifier corresponding to the annotation results. Then, the server can determine, based on the associated data of the test text image, that there are three embedded text points set in the test text image, namely b1, b2, and b3.

[0055] Then, the server can perform targeted text character detection at the corresponding positions in the annotation results based on the location information of the embedded text in the associated data of the test text image, in order to determine the number of embedded texts annotated by the annotator and the number of embedded texts not annotated.

[0056] For example, the detection found that there were text characters marked by the annotator at the placement positions corresponding to b1 and b2 in the annotation results, but no text characters marked by the annotator at the placement position corresponding to b3. Therefore, it can be determined that the number of embedded text marked by the annotator is 2, and the number of embedded text not marked is 1.

[0057] Furthermore, the server can compare the text string of the tracking text in the associated data of the test text image with the text string of the tracking text marked by the annotator, so as to determine and count the tracking text marked correctly by the annotator, and count the number of correctly marked tracking text.

[0058] For example, by comparing the text string of the embedded text b1 in the associated data of the test text image with the text characters marked by the annotator at the corresponding placement position of b1 in the annotation results, if it is found that the text characters marked by the annotator at that position are exactly the same as the text string of b1 (or the character difference value is less than the preset character difference threshold), then b1 is determined to be the embedded text correctly marked by the annotator.

[0059] Conversely, by comparing the text string of the embedded text b2 in the associated data of the test text image with the text characters marked by the annotator at the corresponding placement position of b2 in the annotation results, if it is found that the text characters marked by the annotator at that position are completely different from the text string of b2 (or the character difference value is greater than or equal to the preset character difference threshold), then b2 is determined to be the embedded text correctly marked by the annotator.

[0060] This allows us to determine that the number of correctly labeled tracking text by the annotator is 1.

[0061] Furthermore, the server can calculate evaluation parameters based on the number of tracking texts annotated by the annotator and the number of correctly annotated tracking texts. These parameters include the annotation accuracy rate (Mo), the omission rate (Mm), and the annotation error rate (Me). The annotation quality of the annotator is then quantitatively determined based on these evaluation parameters.

[0062] For example, the annotation accuracy rate is calculated to be Mo = 1 / 3. The omission rate is calculated to be Mn = 1 / 3. The annotation error rate is calculated to be Me = 1 / 3. After obtaining the above three evaluation parameters, the above three evaluation parameters and the preset weight coefficients corresponding to the above three evaluation parameters can be used together to perform a summation operation; the result of the operation is used as the annotation quality of the annotator.

[0063] This allows for the generation of test text images with embedded text that demonstrate good test results. These test text images can then be used to conduct corresponding tests on the annotators, accurately and objectively determining the annotator's annotation quality in a quantitative manner.

[0064] See Figure 6 As shown in the embodiments of this specification, a method for generating test text images is provided. Specifically, this method can be applied to the server side. In specific implementation, the method may include the following:

[0065] S601: Obtain the target text image; wherein the target text image contains at least the text characters to be labeled.

[0066] S602: Preprocess the target text image to obtain the text character data and blank area data in the target text image.

[0067] S603: Based on the text character data and blank area data, set corresponding embedded text in the target text image to obtain the test text image.

[0068] In some embodiments, the aforementioned target text image can be specifically understood as an image or video frame containing text characters to be annotated. See also... Figure 2 As shown. The text characters in the target text image are text characters in image data format.

[0069] In some embodiments, the text character data in the target text image may specifically be a text string that is identified and extracted from the target text image.

[0070] Specifically, the text character data in the acquired target text image can be represented in the following form: w = {w1, w2, ..., wi, ..., wn}. Here, wi can represent the text character numbered i in the text string identified in the target text image.

[0071] Specifically, the blank area data in the target text image can be the location data of the blank areas (or non-text image areas) detected in the target text image.

[0072] In practice, based on the outline shape of the blank area, a rectangle of appropriate size can be used to subdivide the detected blank area in the target image into multiple rectangular blank areas. (See also...) Figure 3 As shown. Furthermore, the position coordinates of the pixels on the diagonal of each rectangular blank area can be used to represent the position data of the corresponding blank area. For example, the x and y coordinates of the pixel at the top left corner and the pixel at the bottom right corner of the blank area can be used to represent the position data of the blank area.

[0073] Specifically, the blank area data in the acquired target text image can be represented in the following form: b = {[x11,y11,x12,y12],[x21,y21,x22,y22],...,[xi1,yi1,xi2,yi2],...,[xn1,yn1,xn2,yn2]}. Here, [xi1,yi1,xi2,yi2] can specifically represent the position data of the i-th blank area (which can be denoted as bi) among the n detected blank areas. Specifically, xi1 and yi1 can be the x and y coordinates of the pixel at the top-left corner of the blank area; xi2 and yi2 can be the x and y coordinates of the pixel at the bottom-right corner of the blank area.

[0074] In some embodiments, the text character data and blank area data in the target text image can be obtained by preprocessing the target text image.

[0075] In some embodiments, the preprocessing of the target text image described above to obtain text character data and blank area data in the target text image may specifically include the following: performing OCR recognition on the target text image to obtain text character data in the target text image; and calling an image detection model to perform blank image detection on the target text image to obtain blank area data in the target text image.

[0076] Specifically, the aforementioned OCR recognition refers to the data processing process of determining the shape of characters in an image by detecting dark and light patterns, and then translating the character shapes into corresponding text characters.

[0077] In some embodiments, a preset OCR recognition model can be invoked to perform OCR recognition processing on the target text image in order to identify and extract the corresponding text character data from the target text image.

[0078] Specifically, the aforementioned preset OCR recognition model can be a well-trained model with high accuracy, or it can be an existing model that has not yet been trained, or a model with lower accuracy but that can support basic OCR recognition functions.

[0079] The text character data in the acquired target text image is allowed to have a preset tolerance for data error.

[0080] In some embodiments, during the OCR recognition of a target text image, if a few text characters in the target text image fail to be recognized by OCR, the corresponding text characters can be determined by querying the system dictionary or system configuration document.

[0081] In some embodiments, the image detection model described above can be a pre-trained image processing model capable of distinguishing between text regions (image regions containing text characters) and blank regions (or non-text regions) in an image, and outputting the location data of the blank regions.

[0082] In practice, the target text image can be used as the model input, fed into the image detection model, and the image detection model can be run to obtain the corresponding model output, which serves as the blank area data in the target text image.

[0083] In some embodiments, the above-described preprocessing of the target text image to obtain text character data and blank area data in the target text image may further include the following: calling a text character classification model to perform graphic classification and recognition of the text characters in the target text image to obtain attribute data of the text characters in the target text image.

[0084] Specifically, the aforementioned text character classification model can be a pre-trained classification model that can detect and determine the attribute data of text characters in an image through image classification.

[0085] The attribute data may specifically include at least one of the following: font, font size, and tilt angle. It should be noted that the attribute data listed above is merely illustrative. In actual implementation, depending on the specific application scenario and processing requirements, the attribute data may also include other attribute data such as color and thickness.

[0086] Specifically, the attribute data of the text characters in the acquired target text image can be represented in the following form: f = {f1, f2, ..., fi, ..., fn}. Here, fi specifically represents the attribute data of the text character with index i in the text character data.

[0087] In some embodiments, during the process of calling a text character classification model to perform graphic classification and recognition of text characters in a target text image, for several text characters that fail to be classified and recognized, the attribute data of the nearest successfully classified text character can be used as the attribute data of the text character that failed to be classified and recognized.

[0088] In some embodiments, the aforementioned embedded text can be understood as text embedded in the target text image for measuring and judging the annotation quality of the target object to be annotated.

[0089] Specifically, the target object mentioned above can be an annotator responsible for data annotation, or an annotation model that can automatically annotate data.

[0090] The aforementioned test text image can be understood as a text image with embedded text points placed within the target text image. Specifically, this test text image can be used to determine the annotation quality of the target object.

[0091] It's important to note that, unlike the original text characters in the target text image, the text string (or the actual value of the tracking text) and its location information in the test text image are known to the server. Therefore, the server can accurately evaluate the annotation results related to the tracking text in the annotation results obtained from the test text image of the target object, based on the text string and its location information, and thus determine the annotation quality of the target object.

[0092] In some embodiments, the above-mentioned setting of corresponding embedded text in the target text image based on the text character data and blank area data may specifically include the following: generating embedded text that matches the text characters in the target text image based on the text character data; determining the required placement position in the target text image based on the blank area data; setting the embedded text at the placement position and recording the associated data of the embedded text.

[0093] Specifically, the associated data of the recorded tracking text may include: the text string of the tracking text, and the location information of the tracking text's placement location, etc.

[0094] Specifically, the associated data of the tracking text can be represented in the following form: DM = {m1, m2, ..., mi ..., mn}. Here, mi can specifically represent the associated data of the tracking text with ID i. Specifically, mi can be represented in the following form: mi = {si = {w1, w2, ..., wt}, bi = [xi1, yi1, xi2, yi2]}. Here, si represents the text string (or real value) of the tracking text with ID i, and bi represents the location information of the placement position of the tracking text with ID i.

[0095] In some embodiments, generating embedded text that matches the text characters in the target text image based on the text character data may specifically include: calling a preset text generation model to process the text character data to generate embedded text that matches the text characters in the target text image.

[0096] Specifically, the aforementioned pre-set text generation model (or text generation service) can refer to a pre-trained data processing model capable of generating text sentences with a high degree of matching, such as similar structure or semantic relevance, based on the input text character data. For example, see [link to relevant documentation]. Figure 4 As shown, the preset text generation model can generate semantically related text sentences b1, b2, and b3 based on the original text character data in the target text image.

[0097] In some embodiments, multiple text characters can be randomly selected from the text character data and combined to obtain a corresponding text string as the embedded text.

[0098] In some embodiments, the above-described method of determining the desired placement position in the target text image based on the blank area data may specifically include: determining multiple blank areas in the target text image based on the blank area data; and selecting blank areas located at the end of a paragraph or between two adjacent paragraphs from the multiple blank areas as the desired placement position.

[0099] In practice, the blank areas above or below text paragraphs can be selected from multiple blank areas as the desired placement locations. For details, please refer to [link to relevant documentation]. Figure 4 As shown.

[0100] By selecting and using the aforementioned blank areas as placement locations to set the embedded text, the difference between the embedded text in the test text image and the original text characters in the target text image can be minimized, and the matching degree can be higher. This makes the embedded text appear more consistent in the test text image, thereby reducing the interference of the differences in the embedded text in the test text image on the subsequent determination of annotation quality.

[0101] In some embodiments, setting embedded text at the deployment location may include the following: obtaining attribute data of text characters in the vicinity of the deployment location; adjusting the relevant attributes of the text characters in the embedded text according to the attribute data of the text characters in the vicinity of the deployment location to obtain the adjusted embedded text; and setting the adjusted embedded text at the deployment location.

[0102] Specifically, for example, if the text characters in the vicinity of the placement location are detected to be in SimSun font, size 12 (small four), then the text characters in the embedded text can also be adjusted to SimSun font, size 12 (small four), resulting in the adjusted embedded text. This adjusted embedded text is then set as image data at the placement location. This makes the embedded text in the test text image more consistent and coordinated with the surrounding original text characters, further reducing interference with subsequent determination of annotation quality.

[0103] Specifically, the aforementioned proximity range may include: a range of areas where the distance from the deployment location is less than or equal to a preset distance threshold.

[0104] In some embodiments, after setting the embedded text at the location, the method may further include: performing character alignment processing on the set embedded text based on the text characters in the vicinity of the placement location.

[0105] In addition, the spacing between text characters in the embedded text can be adjusted based on the text characters in the vicinity of the placement location, and appropriate punctuation marks can be set for the embedded text.

[0106] Through the above embodiments, the difference between the embedded text and the adjacent original text characters can be reduced, and the text can be made to appear more coordinated and matched in the test text image, thereby obtaining a test text image with relatively better test results.

[0107] In some embodiments, after obtaining the test text image, the method may further include the following: using the test text image to determine the annotation quality of the target object.

[0108] In some embodiments, the above-described method of using the test text image to determine the annotation quality of the target object may include the following: sending the test text image to the target object; wherein the target object annotates the text characters in the test text image to obtain corresponding annotation results; receiving the annotation results; and determining the annotation quality of the target object based on the annotation results and the associated data of the embedded text.

[0109] The above annotation results can be specifically represented in the following form: DL={Lm1,Lm2,...,Lmi,...,Lmk}. Specifically, Lmi can be represented as the annotation result numbered i in the annotation results.

[0110] In this embodiment, the target object is unaware of which text strings in the test text image belong to the original text characters and which belong to the embedded text. Therefore, when annotating the test text image, the target object does not distinguish between the original text strings and the embedded text, but annotates all the identified text characters. For example, the target object can annotate all the identified text characters in the text test image with the corresponding text data format as the annotation result for the test text image.

[0111] The server can receive the annotation results after the target object is annotated, and then evaluate the annotation results of the tracking text based on the associated data of the tracking text to determine the annotation quality of the target object.

[0112] In some embodiments, the annotation quality of the target object can be determined based on the annotation results and the associated data of the embedded text in the following manner: Based on the associated data of the embedded text, determine the text string of the embedded text in the test text image and the location information of the embedded text placement position; based on the annotation results, the text string of the embedded text, and the location information of the embedded text placement position, count the number of embedded texts annotated on the target object and the number of correctly annotated embedded texts; based on the number of embedded texts annotated on the target object and the number of correctly annotated embedded texts, determine the annotation quality of the target object.

[0113] In some embodiments, the number of tracked text annotations on the target object can be denoted as T. Simultaneously, based on T and the associated data of the tracked text, the number of unannotated tracked text annotations on the target object can be calculated and denoted as Tb. Furthermore, the number of correctly annotated tracked text annotations on the target object can be denoted as Te.

[0114] Furthermore, based on the number of tracked texts labeled on the target object and the number of correctly labeled tracked texts, evaluation parameters such as the labeling accuracy rate (which can be denoted as Mo), the missing labeling rate (which can be denoted as Mm), and the labeling error rate (which can be denoted as Me) of the tracked texts can be calculated.

[0115] Next, based on the preset weighting coefficients, the annotation quality of the target object can be calculated by comprehensively utilizing the annotation accuracy rate, the missing annotation rate, and the annotation error rate of the target object's embedded text.

[0116] The above-mentioned labeling accuracy rate can be calculated using the following formula: Mo = 1 - (Tb + Te) / (T) * 100%. The above-mentioned omission rate can be calculated using the following formula: Mm = Tb / (T) * 100%. The above-mentioned labeling error rate can be calculated using the following formula: Me = Te / T * 100%.

[0117] In some embodiments, multiple different test text images can be generated and sent to the target object for annotation, resulting in multiple annotation results from the target object for the multiple test text images. Then, based on these multiple annotation results and the associated data of the embedded text in the multiple test text images, the annotation quality of the target object can be determined more accurately.

[0118] In some embodiments, after determining the annotation quality of the target object, the method may further include: determining whether the annotation quality of the target object meets preset quality requirements. If it is determined that the annotation quality of the target object does not meet the preset quality requirements, a prompt message can be generated for the target object. This allows for more accurate and objective identification of target objects with poor annotation quality that do not meet the preset quality requirements; and by generating corresponding prompt messages, targeted prompts can be given and improvements can be encouraged for the target object to be made, thereby improving the annotation quality.

[0119] In some embodiments, the above-described method can be used to test multiple different target objects to determine the annotation quality of each target object. Furthermore, a reliability weighting coefficient for the annotated sample data of each target object can be determined based on its annotation quality.

[0120] Specifically, for example, if the annotation quality of target object A is higher than that of target object B, then the reliability weight coefficient of the annotated sample data provided by target object A can be set to a relatively large value, while the reliability weight coefficient of the annotated sample data provided by target object B can be set to a relatively small value.

[0121] In this way, when training the OCR recognition model using labeled sample data provided by multiple target objects, the aforementioned reliability weight coefficients can be combined to effectively distinguish and use labeled sample data provided by different target objects. This allows the model to selectively increase the learning of labeled sample data provided by target objects with high labeled quality during the training process, while reducing the learning of labeled sample data provided by target objects with poor labeled quality, thereby improving the model training accuracy.

[0122] As can be seen from the above, the method for generating test text images based on the embodiments of this specification can first preprocess the target text image to be labeled to obtain the text character data and blank area data in the target text image; then, according to the text character data and blank area data, corresponding embedded text can be set in the target text image, thereby efficiently and automatically generating test text images with embedded text that has a high degree of matching with the original text characters and good test results. Subsequently, the above test text images can be used to determine the labeling quality of the target object to be labeled more accurately.

[0123] See Figure 7 As shown in the embodiments of this specification, a method for determining annotation quality is also provided. In specific implementation, this method may include the following:

[0124] S701: Generate a corresponding test text image based on the target text image; wherein, the test text image contains embedded text; the embedded text is generated based on the text character data in the target text image;

[0125] S702: Using the test text image, determine the annotation quality of the target object.

[0126] In some embodiments, the above-mentioned generation of a corresponding test text image based on a target text image may specifically include the following: obtaining a target text image; wherein the target text image contains at least text characters to be labeled; preprocessing the target text image to obtain text character data and blank area data in the target text image; setting corresponding embedded text in the target text image based on the text character data and blank area data to obtain a test text image; and recording the associated data of the embedded text.

[0127] Since the embedded text is generated based on the original text character data in the target text image, the embedded text has little difference and a high degree of matching compared with the surrounding original text characters. It appears more coordinated and uniform in the test text image, which can effectively reduce the interference and impact of the embedded text on the subsequent determination of the annotation quality of the target object.

[0128] In some embodiments, the associated data of the embedded text may include at least: the text string of the embedded text, and the location information of the embedded text placement location, etc.

[0129] In some embodiments, the above-described method of using the test text image to determine the annotation quality of the target object may include the following: sending the test text image to the target object; wherein the target object annotates the text characters in the test text image to obtain corresponding annotation results; receiving the annotation results; and determining the annotation quality of the target object based on the annotation results and the associated data of the embedded text.

[0130] In some embodiments, determining the annotation quality of the target object based on the annotation results and the associated data of the embedded text can specifically include the following: determining the text string of the embedded text in the test text image and the location information of the embedded text placement position based on the associated data of the embedded text; counting the number of embedded texts annotated on the target object and the number of correctly annotated embedded texts based on the annotation results, the text string of the embedded text, and the location information of the embedded text placement position; and determining the annotation quality of the target object based on the number of embedded texts annotated on the target object and the number of correctly annotated embedded texts.

[0131] As can be seen from the above, based on the method for determining annotation quality provided in the embodiments of this specification, test text images with good test results can be generated first, and then the annotation quality of the target object can be determined more accurately and objectively using the test text images.

[0132] This specification also provides a server, including a processor and a memory for storing processor-executable instructions. Specifically, the processor can perform the following steps according to the instructions: acquiring a target text image; wherein the target text image contains at least text characters to be labeled; preprocessing the target text image to obtain text character data and blank area data in the target text image; and setting corresponding embedded text in the target text image based on the text character data and blank area data to obtain a test text image.

[0133] To execute the above instructions more accurately, please refer to... Figure 8This specification also provides another specific server in the embodiments, wherein the server includes a network communication port 801, a processor 802 and a memory 803, and the above structures are connected by internal cables so that the various structures can perform specific data interaction.

[0134] Specifically, the network communication port 801 can be used to acquire a target text image; wherein the target text image contains at least text characters to be labeled.

[0135] The processor 802 can be used to preprocess the target text image to obtain text character data and blank area data in the target text image; and to set corresponding embedded text in the target text image according to the text character data and blank area data to obtain a test text image.

[0136] The memory 803 can be used to store the corresponding instruction program.

[0137] In this embodiment, the network communication port 801 can be a virtual port bound to different communication protocols, thereby enabling the sending or receiving of different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. Furthermore, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM or CDMA; it can also be a Wi-Fi chip; or it can be a Bluetooth chip.

[0138] In this embodiment, the processor 802 can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. This specification is not limiting.

[0139] In this embodiment, the memory 803 may include multiple layers. In a digital system, anything that can store binary data can be a memory. In an integrated circuit, a circuit with storage function but no physical form is also called a memory, such as RAM, FIFO, etc. In a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0140] This specification also provides another server, including a processor and a memory for storing processor-executable instructions. In a specific implementation, the processor can perform the following steps according to the instructions: generating a corresponding test text image based on a target text image; wherein, the test text image contains embedded text; the embedded text is generated based on the text character data in the target text image; and using the test text image to determine the annotation quality of the target object.

[0141] This specification also provides a computer storage medium based on the above-described method for generating test text images. The computer storage medium stores computer program instructions that, when executed, perform the following: acquiring a target text image; wherein the target text image contains at least text characters to be labeled; preprocessing the target text image to obtain text character data and blank area data in the target text image; and setting corresponding embedded text in the target text image based on the text character data and blank area data to obtain a test text image.

[0142] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), cache, hard disk drive (HDD), or memory card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.

[0143] This specification also provides a computer storage medium based on the above-described method for determining annotation quality. The computer storage medium stores computer program instructions that, when executed, perform the following: generating a corresponding test text image based on a target text image; wherein the test text image contains embedded text; and using the test text image to determine the annotation quality of the target object.

[0144] In this embodiment, the specific functions and effects implemented by the program instructions stored in the computer storage medium can be explained in comparison with other implementation methods, and will not be repeated here.

[0145] See Figure 9 As shown, at the software level, this specification also provides a device for generating test text images, which may specifically include the following structural modules:

[0146] The acquisition module 901 can be specifically used to acquire a target text image; wherein the target text image contains at least text characters to be labeled;

[0147] The preprocessing module 902 can be used to preprocess the target text image to obtain the text character data and blank area data in the target text image;

[0148] The setting module 903 can be used to set corresponding embedded text in the target text image based on the text character data and blank area data to obtain a test text image.

[0149] In some embodiments, when the preprocessing module 902 is specifically implemented, the target text image can be preprocessed in the following manner to obtain text character data and blank area data in the target text image: perform OCR recognition on the target text image to obtain text character data in the target text image; call an image detection model to perform blank image detection on the target text image to obtain blank area data in the target text image.

[0150] In some embodiments, the preprocessing module 902 described above can also be used to call a text character classification model to perform graphic classification and recognition of text characters in a target text image in order to obtain attribute data of text characters in the target text image; wherein, the attribute data includes at least one of the following: font, font size, and tilt angle.

[0151] In some embodiments, when the setting module 903 is specifically implemented, it can generate embedded text that matches the text characters in the target text image based on the text character data; determine the required placement position in the target text image based on the blank area data; set the embedded text at the placement position; and record the associated data of the embedded text.

[0152] In some embodiments, when the above-mentioned setting module 903 is specifically implemented, it can call a preset text generation model to process the text character data in order to generate embedded text that matches the text characters in the target text image.

[0153] In some embodiments, when the setting module 903 is specifically implemented, it can determine multiple blank areas in the target text image based on the blank area data; and select blank areas located at the end of a paragraph or between two adjacent paragraphs from the multiple blank areas as the layout positions that meet the requirements.

[0154] In some embodiments, when the setting module 903 is specifically implemented, the embedded text can be set at the deployment location in the following manner: obtaining attribute data of text characters in the vicinity of the deployment location; adjusting the relevant attributes of the text characters in the embedded text according to the attribute data of the text characters in the vicinity of the deployment location to obtain the adjusted embedded text; and setting the adjusted embedded text at the deployment location.

[0155] In some embodiments, the apparatus further includes a quality determination module, which can be used to determine the annotation quality of the target object using the test text image after obtaining the test text image.

[0156] In some embodiments, when the quality determination module is specifically implemented, the test text image can be sent to the target object; wherein, the target object annotates the text characters in the test text image to obtain corresponding annotation results; receives the annotation results; and determines the annotation quality of the target object based on the annotation results and the association data of the embedded text.

[0157] See Figure 10 As shown in the embodiments of this specification, an apparatus for determining the quality of annotations is also provided, which may specifically include the following structural modules:

[0158] The generation module 1001 is specifically used to generate a corresponding test text image based on the target text image; wherein, the test text image contains embedded text; the embedded text is generated based on the text character data in the target text image.

[0159] The determination module 1002 can be used to determine the annotation quality of the target object using the test text image.

[0160] In some embodiments, when the above-mentioned generation module 1001 is specifically implemented, it can obtain a target text image; wherein the target text image contains at least text characters to be labeled; preprocess the target text image to obtain text character data and blank area data in the target text image; set corresponding embedded text in the target text image according to the text character data and blank area data to obtain a test text image; and record the associated data of the embedded text.

[0161] In some embodiments, the associated data of the embedded text may include at least: the text string of the embedded text, and the location information of the embedded text's placement location, etc.

[0162] In some embodiments, when the determination module 1002 is specifically implemented, the test text image can be sent to the target object; wherein, the target object annotates the text characters in the test text image to obtain corresponding annotation results; receives the annotation results; and determines the annotation quality of the target object based on the annotation results and the association data of the embedded text.

[0163] In some embodiments, when the determining module 1002 is specifically implemented, it can determine the text string of the embedded text in the test text image and the location information of the embedded text placement position based on the associated data of the embedded text; based on the annotation results, the text string of the embedded text, and the location information of the embedded text placement position, it can count the number of embedded texts annotated on the target object and the number of correctly annotated embedded texts; based on the number of embedded texts annotated on the target object and the number of correctly annotated embedded texts, it can determine the annotation quality of the target object.

[0164] It should be noted that the units, devices, or modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above devices are described by dividing them into various modules according to their functions. Of course, in implementing this specification, the functions of each module can be implemented in one or more software and / or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection between the devices or units shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0165] As can be seen from the above, the related apparatus provided in the embodiments of this specification can generate test text images with embedded text that have good test results, and use the test text images to perform corresponding tests on the target object to accurately and objectively determine the annotation quality of the target object in a quantitative manner.

[0166] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or client product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0167] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0168] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0169] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this specification can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of this specification can essentially be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of this specification.

[0170] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. This specification can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0171] Although this specification has been described by way of examples, those skilled in the art will recognize that many variations and modifications are possible without departing from the spirit of this specification, and it is intended that the appended claims cover such variations and modifications without departing from the spirit of this specification.

Claims

1. A method for generating test text images, comprising: Obtain the target text image; wherein the target text image contains at least the text characters to be labeled; Preprocess the target text image to obtain the text character data and blank area data in the target text image; Based on the text character data and blank area data, corresponding embedded text is set in the target text image to obtain a test text image; this includes: generating embedded text that matches the text characters in the target text image based on the text character data; determining a suitable placement position in the target text image based on the blank area data; obtaining attribute data of text characters in the vicinity of the placement position; adjusting the relevant attributes of the text characters in the embedded text based on the attribute data of the text characters in the vicinity of the placement position to obtain the adjusted embedded text; setting the adjusted embedded text at the placement position and recording the associated data of the embedded text; the vicinity includes: a range of areas whose distance from the placement position is less than or equal to a preset distance threshold.

2. The method according to claim 1, wherein preprocessing the target text image to obtain text character data and blank area data in the target text image includes: Perform OCR recognition on the target text image to obtain the text character data in the target text image; The image detection model is invoked to perform blank image detection on the target text image in order to obtain blank area data in the target text image.

3. The method according to claim 2, further comprising preprocessing the target text image: A text character classification model is invoked to perform graphic classification and recognition of text characters in a target text image to obtain attribute data of the text characters in the target text image; wherein, the attribute data includes at least one of the following: font, font size, and tilt angle.

4. The method according to claim 1, wherein generating embedded text that matches the text characters in the target text image based on the text character data, comprising: The preset text generation model is invoked to process the text character data in order to generate embedded text that matches the text characters in the target text image.

5. The method according to claim 1, wherein determining the required placement position in the target text image based on the blank area data includes: Based on the blank area data, multiple blank areas are identified in the target text image; From the multiple blank areas, select the blank areas located at the end of a segment or between two adjacent segments as the qualified placement positions.

6. The method according to claim 1, further comprising, after obtaining the test text image: The quality of the annotation of the target object is determined using the test text image.

7. The method according to claim 6, wherein the annotation quality of the target object is determined using the test text image, comprising: The test text image is sent to the target object; wherein the target object annotates the text characters in the test text image to obtain the corresponding annotation results; Receive annotation results; Based on the annotation results and the associated data of the embedded text, the annotation quality of the target object is determined.

8. A method for determining annotation quality, comprising: A corresponding test text image is generated based on the target text image. The test text image contains embedded text, which is generated based on the text character data in the target text image. The test text image is generated as follows: the target text image is preprocessed to obtain text character data and blank area data; embedded text matching the text characters in the target text image is generated based on the text character data; a suitable placement position is determined in the target text image based on the blank area data; attribute data of text characters in the vicinity of the placement position is obtained; the relevant attributes of the text characters in the embedded text are adjusted based on the attribute data of the text characters in the vicinity of the placement position to obtain the adjusted embedded text; the adjusted embedded text is placed at the placement position, and the associated data of the embedded text is recorded; the vicinity includes a range of areas whose distance from the placement position is less than or equal to a preset distance threshold. The quality of the annotation of the target object is determined using the test text image.

9. The method according to claim 8, wherein the associated data of the embedded text includes at least: The text string of the embedded text, and the location information of the embedded text placement position.

10. The method according to claim 9, wherein the annotation quality of the target object is determined using the test text image, comprising: The test text image is sent to the target object; wherein the target object annotates the text characters in the test text image to obtain the corresponding annotation results; Receive annotation results; Based on the annotation results and the associated data of the embedded text, the annotation quality of the target object is determined.

11. The method according to claim 10, wherein determining the annotation quality of the target object based on the annotation results and the associated data of the embedded text, includes: Based on the associated data of the embedded text, determine the text string of the embedded text in the test text image, as well as the location information of the embedded text placement position; Based on the annotation results, the text string of the embedded text, and the location information of the embedded text placement position, count the number of embedded texts annotated on the target object and the number of correctly annotated embedded texts. The annotation quality of the target object is determined by the number of tracking texts marked on the target object and the number of correctly marked tracking texts.

12. An apparatus for generating test text images, comprising: An acquisition module is used to acquire a target text image; wherein the target text image contains at least text characters to be labeled; The preprocessing module is used to preprocess the target text image to obtain the text character data and blank area data in the target text image; The setting module is used to set corresponding embedded text in the target text image based on the text character data and blank area data, so as to obtain the test text image. Specifically, the setting module is used to: generate embedded text that matches the text characters in the target text image based on the text character data; determine a suitable placement position in the target text image based on the blank area data; obtain attribute data of text characters in the vicinity of the placement position; adjust the relevant attributes of the text characters in the embedded text based on the attribute data of the text characters in the vicinity of the placement position to obtain the adjusted embedded text; set the adjusted embedded text at the placement position and record the associated data of the embedded text; the vicinity includes: a range of areas whose distance from the placement position is less than or equal to a preset distance threshold.

13. A device for determining the quality of a mark, comprising: A generation module is used to generate a corresponding test text image based on a target text image. The test text image contains embedded text, which is generated based on text character data in the target text image. The test text image is generated as follows: the target text image is preprocessed to obtain text character data and blank area data; embedded text matching the text characters in the target text image is generated based on the text character data; a suitable placement position is determined in the target text image based on the blank area data; attribute data of text characters in the vicinity of the placement position is obtained; the relevant attributes of the text characters in the embedded text are adjusted based on the attribute data of the text characters in the vicinity of the placement position to obtain adjusted embedded text; the adjusted embedded text is placed at the placement position, and the associated data of the embedded text is recorded; the vicinity includes a range of areas whose distance from the placement position is less than or equal to a preset distance threshold. The determination module is used to determine the annotation quality of the target object using the test text image.

14. A server comprising a processor and a memory for storing processor-executable instructions, wherein the processor, when executing the instructions, implements the steps of the method of any one of claims 1 to 7 or 8 to 11.

15. A computer-readable storage medium having stored thereon computer instructions that, when executed, perform the steps of the method according to any one of claims 1 to 7 or 8 to 11.

Images