Image slice based labeling method and device, electronic device, and storage medium

Abstract

The application discloses an image slice-based labeling method and device, electronic equipment and storage medium, and relates to the field of artificial intelligence, wherein the labeling method comprises the following steps: receiving an image slice labeling request, identifying each image slice in an image slice set by using a first identification model to obtain a first identification result, identifying each image slice in the image slice set by using a second identification model to obtain a second identification result, establishing a screening degree model based on a first confidence degree, first identification character information, a second confidence degree and second identification character information, determining a screening degree value of each image slice based on the screening degree model, screening the image slice set based on the screening degree value to obtain a target image slice set, and labeling each target image slice in the target image slice set. The application solves the technical problem of low efficiency and high cost in the prior art that relies only on manual labeling of a large number of image slices.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to an image slice based labeling method and device, electronic equipment and storage medium. BACKGROUND

[0002] Currently, for image text recognition field labeling, it can be divided into two parts: one part is the labeling of the positioning information of the text in the image (i.e. forming a positioning box), and the other part is the labeling of the text in the positioning box. The process of labeling the text in the positioning box can be called the process of recognizing information. The positioning information is mainly used to train the positioning model in the image text recognition model, and the recognition information is mainly used to train the recognition model. The recognition information is mainly obtained by labeling the image slice with text content positioned by the positioning box.

[0003] The training of the image text recognition model is usually to first train the positioning model with the positioning information, and then train the recognition model with the recognition information to recognize the text in the positioning box. The training of the positioning model and the recognition model is usually separated, and the training data used for training the recognition model is the data labeled on the image slice. However, training the recognition model, especially large recognition model, often needs to cover enough training data with annotated character amount, and the workload of labeling these characters is very large.

[0004] In related technologies, the image slice labeling method is usually labeled by artificial method. Even though the recognition information labeling of the image slice is relatively simple than other labeling, it only needs to label the text on the image slice, but when the amount of image slices is large, the labeling work is still very huge. For example, when one million image slices are needed for recognition model training, the labeling work is very large, especially when the number of characters to be trained in image text recognition increases, the amount of image slice data increases exponentially. Current manual labeling is difficult to meet the data demand, and there is a problem of low efficiency and high cost of manual labeling.

[0005] For the above problems, no effective solution has been proposed so far. SUMMARY

[0006] The embodiments of the present application provide an image slice based labeling method and device, electronic equipment and storage medium to at least solve the technical problems of low efficiency and high cost of relying only on manual labeling of a large number of image slices in related technologies.

[0007] According to an aspect of some embodiments of the present application, there is provided a method for image slice labeling, comprising: receiving an image slice labeling request, wherein the image slice labeling request comprises at least: a set of image slices; identifying each image slice in the set of image slices using a first recognition model to obtain a first recognition result, and identifying each image slice in the set of image slices using a second recognition model to obtain a second recognition result, wherein the first recognition model and the second recognition model are pre-trained recognition models, the first recognition result comprises at least: a first confidence, first recognized character information, and the second recognition result comprises at least: a second confidence, second recognized character information; establishing a screening degree model based on the first confidence, the first recognized character information, the second confidence, and the second recognized character information; determining a screening degree value of each image slice based on the screening degree model, and screening the set of image slices based on the screening degree value to obtain a set of target image slices; and labeling each target image slice in the set of target image slices.

[0008] Optionally, the step of establishing a screening degree model based on the first confidence, the first recognized character information, the second confidence, and the second recognized character information comprises: determining a first matching degree output by the first recognition model based on the first recognized character information and the second recognized character information; determining a second matching degree output by the second recognition model based on the first recognized character information and the second recognized character information, wherein the first matching degree and the second matching degree are used to measure the consistency between the first recognition result and the second recognition result; determining a preset first parameter and a preset second parameter; and establishing the screening degree model based on the first confidence, the second confidence, the first matching degree, the second matching degree, the preset first parameter, and the preset second parameter.

[0009] Optionally, the step of determining a first matching degree output by the first recognition model based on the first recognized character information and the second recognized character information comprises: matching the first recognized character information and the second recognized character information to obtain a number of identical characters; determining a total number of first characters recognized by the first recognition model based on the first recognized character information; calculating a first ratio between the number of identical characters and the total number of first characters; and representing the first ratio as the first matching degree.

[0010] Optionally, the step of determining the second matching degree output by the second recognition model based on the first recognized character information and the second recognized character information includes: matching the first recognized character information and the second recognized character information to obtain the number of identical characters; determining the total number of second characters recognized by the second recognition model based on the second recognized character information; calculating a second ratio between the number of identical characters and the total number of second characters; and representing the second ratio as the second matching degree.

[0011] Optionally, the step of filtering the image slice set based on the filtering degree value to obtain a target image slice set includes: sorting the filtering degree values ​​to obtain a sorting result; and filtering the image slices indicated by the filtering degree values ​​within a preset range based on the sorting result to obtain the target image slice set, wherein all the filtering degree values ​​within the preset range are greater than the filtering degree threshold.

[0012] Optionally, the step of annotating each target image slice in the target image slice set includes: obtaining initial character information for annotating each target image slice based on the first identification character information and the second identification character information; adjusting the corresponding initial character information according to each target image slice to complete the annotation of the target image slice, wherein each annotated target image slice corresponds to character information.

[0013] Optionally, after labeling each target image slice in the target image slice set, the method further includes: for the image slices whose screening degree value is less than or equal to the screening degree threshold, using the character information in the first recognition result or the second recognition result as the character information of the image slice; representing all the image slices with character information and all the target image slices as a training sample set; and using the training sample set to train the recognition model to be trained until the recognition accuracy of the recognition model to be trained is greater than a preset accuracy threshold.

[0014] According to another aspect of the present invention, an image slice annotation apparatus is also provided, comprising: a receiving unit for receiving an image slice annotation request, wherein the image slice annotation request includes at least: an image slice set; an identification unit for identifying each image slice in the image slice set using a first identification model to obtain a first identification result, and identifying each image slice in the image slice set using a second identification model to obtain a second identification result, wherein the first identification model and the second identification model are pre-trained identification models, the first identification result includes at least: a first confidence level and first identification character information, and the second identification result includes at least: a second confidence level and second identification character information; an establishment unit for establishing a screening degree model based on the first confidence level, the first identification character information, the second confidence level, and the second identification character information; a screening unit for determining a screening degree value for each image slice based on the screening degree model, and screening the image slice set based on the screening degree value to obtain a target image slice set; and an annotation unit for annotating each target image slice in the target image slice set.

[0015] Optionally, the establishment unit includes: a first determining module, configured to determine a first matching degree output by the first recognition model based on the first recognition character information and the second recognition character information; a second determining module, configured to determine a second matching degree output by the second recognition model based on the first recognition character information and the second recognition character information, wherein the first matching degree and the second matching degree are used to measure the consistency between the first recognition result and the second recognition result; a third determining module, configured to determine a preset first parameter and a preset second parameter; and a first establishing module, configured to establish the screening degree model based on the first confidence level, the second confidence level, the first matching degree, the second matching degree, the preset first parameter, and the preset second parameter.

[0016] Optionally, the first determining module includes: a first matching submodule, configured to match the first identified character information and the second identified character information to obtain the number of identical characters; a first determining submodule, configured to determine the total number of first characters identified by the first identification model based on the first identified character information; a first calculation submodule, configured to calculate a first ratio between the number of identical characters and the total number of first characters; and a first characterization submodule, configured to characterize the first ratio as the first matching degree.

[0017] Optionally, the second determining module includes: a second matching submodule, used to match the first identified character information and the second identified character information to obtain the number of identical characters; a second determining submodule, used to determine the total number of second characters identified by the second identification model based on the second identified character information; a second calculation submodule, used to calculate a second ratio between the number of identical characters and the total number of second characters; and a second characterization submodule, used to characterize the second ratio as the second matching degree.

[0018] Optionally, the filtering unit includes: a first sorting module, used to sort the filtering degree values ​​to obtain a sorting result; and a first filtering module, used to filter the image slices indicated by the filtering degree values ​​within a preset range based on the sorting result to obtain the target image slice set, wherein all the filtering degree values ​​within the preset range are greater than the filtering degree threshold.

[0019] Optionally, the annotation unit includes: a first output module, configured to obtain initial character information for annotating each target image slice based on the first identification character information and the second identification character information; and a first adjustment module, configured to adjust the corresponding initial character information according to each target image slice to complete the annotation of the target image slice, wherein each target image slice that has been annotated corresponds to character information.

[0020] Optionally, the annotation device further includes: a first assignment module, configured to, after annotating each target image slice in the target image slice set, use the character information in the first recognition result or the second recognition result as the character information of the image slice for the image slice whose screening degree value is less than or equal to the screening degree threshold; a first representation module, configured to represent all the image slices with character information and all the target image slices as a training sample set; and a first training module, configured to train the recognition model to be trained using the training sample set until the recognition accuracy of the recognition model to be trained is greater than a preset accuracy threshold.

[0021] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the above-described image tile-based annotation method.

[0022] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the above-described image tile-based annotation method.

[0023] In this disclosure, an image slice annotation request is received, each image slice in an image slice set is identified using a first recognition model to obtain a first recognition result, and each image slice in the image slice set is identified using a second recognition model to obtain a second recognition result. A screening degree model is established based on a first confidence level, first recognition character information, a second confidence level, and second recognition character information. Based on the screening degree model, a screening degree value for each image slice is determined, and the image slice set is screened based on the screening degree value to obtain a target image slice set. Each target image slice in the target image slice set is then annotated. In this disclosure, a screening degree model can be established based on the first confidence level and first recognition character information identified by the first recognition model for the image slice, and the second confidence level and second recognition character information identified by the second recognition model for the image slice. The screening degree model determines the screening degree value for each image slice, and then the target image slices to be annotated are selected based on the screening degree value. This reduces the number of image slices to be annotated, effectively improves annotation efficiency, reduces costs, and thus solves the technical problem in related technologies where relying solely on manual annotation of a large number of image slices is inefficient and costly. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0025] Figure 1 This is a flowchart of an optional image tile-based annotation method according to an embodiment of the present invention;

[0026] Figure 2 This is a flowchart of an optional screening model according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of an optional image slice identification information annotation process according to an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of an optional image slice-based annotation device according to an embodiment of the present invention;

[0029] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) for an image slicing-based annotation method according to an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] To facilitate understanding of the present invention by those skilled in the art, some terms or nouns involved in the various embodiments of the present invention are explained below:

[0033] Confidence level: A quantitative value used to indicate the likelihood that a model believes its output is correct.

[0034] It should be noted that the image slice-based annotation method and apparatus disclosed herein can be used in the field of artificial intelligence for the annotation of image slices, and can also be used in any field other than artificial intelligence for the annotation of image slices. This disclosure does not limit the application field of the image slice-based annotation method and apparatus.

[0035] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent information from the aforementioned user or organization.

[0036] The following embodiments of the present invention can be applied to various systems / applications / devices for annotating image slices. The present invention constructs a screening model by fusing the confidence scores output by different recognition models and the matching degree between the models. This screening model can determine the image slices that need to be annotated, effectively reducing the workload of annotation.

[0037] The present invention will now be described in detail with reference to various embodiments.

[0038] Example 1

[0039] According to an embodiment of the present invention, an embodiment of an image slicing annotation method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0040] Figure 1 This is a flowchart of an optional image tile-based annotation method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0041] Step S101: Receive an image tile annotation request, wherein the image tile annotation request includes at least: a set of image tiles.

[0042] Step S102: Use a first recognition model to recognize each image slice in the image slice set to obtain a first recognition result, and use a second recognition model to recognize each image slice in the image slice set to obtain a second recognition result. The first recognition model and the second recognition model are both pre-trained recognition models. The first recognition result includes at least: a first confidence level and first recognition character information. The second recognition result includes at least: a second confidence level and second recognition character information.

[0043] Step S103: Establish a screening model based on the first confidence level, the first identification character information, the second confidence level, and the second identification character information.

[0044] Step S104: Based on the screening degree model, determine the screening degree value of each image slice, and based on the screening degree value, screen the image slice set to obtain the target image slice set.

[0045] Step S105: Label each target image slice in the target image slice set.

[0046] Through the above steps, an image slice annotation request can be received. A first recognition model is used to identify each image slice in the image slice set, obtaining a first recognition result. A second recognition model is then used to identify each image slice in the image slice set, obtaining a second recognition result. Based on a first confidence level, first recognition character information, a second confidence level, and second recognition character information, a screening degree model is established. Based on the screening degree model, a screening degree value for each image slice is determined. Based on the screening degree value, the image slice set is screened to obtain a target image slice set. Each target image slice in the target image slice set is then labeled. In this embodiment of the invention, a screening degree model can be established based on the first confidence level and first recognition character information identified by the first recognition model for the image slice, and the second confidence level and second recognition character information identified by the second recognition model for the image slice. The screening degree model determines the screening degree value for each image slice. Then, based on the screening degree value, the target image slices that need to be labeled are selected. This reduces the number of image slices that need to be labeled, effectively improves labeling efficiency, reduces costs, and thus solves the technical problem in related technologies where relying solely on manual labeling of a large number of image slices results in low efficiency and high cost.

[0047] The embodiments of the present invention will now be described in detail with reference to the steps described above.

[0048] Step S101: Receive an image tile annotation request, wherein the image tile annotation request includes at least: a set of image tiles.

[0049] In this embodiment of the invention, an image slice annotation request may be received first, which carries a set of image slices that need to be annotated.

[0050] Step S102: Use a first recognition model to recognize each image slice in the image slice set to obtain a first recognition result, and use a second recognition model to recognize each image slice in the image slice set to obtain a second recognition result. The first recognition model and the second recognition model are both pre-trained recognition models. The first recognition result includes at least: a first confidence level and first recognition character information. The second recognition result includes at least: a second confidence level and second recognition character information.

[0051] In this embodiment of the invention, two pre-trained recognition models (i.e., a first recognition model and a second recognition model, both of which are pre-trained recognition models) can be used to recognize each image slice in the image slice set, and a first recognition result and a second recognition result can be obtained respectively. The first recognition result includes: a first confidence level conA (i.e., the probability of correct recognition) and a first recognition character information (i.e., the recognized characters and the number of characters) for each image slice. The second recognition result includes: a second confidence level conB (i.e., the probability of correct recognition) and a second recognition character information (i.e., the recognized characters and the number of characters) for each image slice.

[0052] Step S103: Establish a screening model based on the first confidence level, the first identification character information, the second confidence level, and the second identification character information.

[0053] Optionally, the step of establishing a screening degree model based on a first confidence level, first identified character information, a second confidence level, and second identified character information includes: determining a first matching degree output by a first identification model based on the first identified character information and the second identified character information; determining a second matching degree output by a second identification model based on the first identified character information and the second identified character information, wherein the first matching degree and the second matching degree are used to measure the consistency between the first identification result and the second identification result; determining a preset first parameter and a preset second parameter; and establishing a screening degree model based on the first confidence level, the second confidence level, the first matching degree, the second matching degree, the preset first parameter, and the preset second parameter.

[0054] In this embodiment of the invention, a screening degree model can be established based on the confidence level and recognized character information in the recognition results of two recognition models. Specifically, based on the first and second recognized character information, a first matching degree MA output by the first recognition model and a second matching degree MB output by the second recognition model are determined. The matching degree M is a measure of the consistency between the recognition results of the two recognition models (i.e., the first and second matching degrees are used to measure the consistency between the first and second recognition results). Then, a preset first parameter ε1 and a preset second parameter ε2 are determined. Finally, based on the first confidence level, the second confidence level, the first matching degree, the second matching degree, the preset first parameter, and the preset second parameter, a screening degree model Ch is established. The screening degree model Ch is represented as follows:

[0055] Ch=-(1-conA+ε1)log(MA+ε2)-(1-conB+ε1)log(MB+ε2);

[0056] Where conA is the first confidence level, conB is the second confidence level, MA is the first matching degree, MB is the second matching degree, ε1 is the preset first parameter, and ε2 is the preset second parameter.

[0057] In this embodiment, the two parameters ε1 and ε2 are pre-set very small numbers to prevent the numbers in the 1-con term and the log term from being 0. If the number in the log term is 0, the result of the log calculation will be infinite, making further calculations impossible. Similarly, if the 1-con term is 0, the subsequent log term will not be calculated. If both recognition models have a confidence level of 1 for the image slice, the calculated screening degree will be 0, even though their matching degrees may not be consistent. This is detrimental to the screening and labeling of image slices.

[0058] In this embodiment, the screening model integrates the confidence score and the matching information from the two models. By utilizing the characteristic that the log function grows rapidly in the (0, 1) interval, it can better distinguish the matching scores. Furthermore, 1-conA and 1-conB can select samples with low confidence scores. Therefore, the screening model established in this embodiment can filter out image slices with low matching scores and low confidence scores.

[0059] Figure 2 This is a flowchart of an optional screening model according to an embodiment of the present invention, such as... Figure 2 As shown, it includes the following steps:

[0060] Step S201: Based on the first recognized character information and the second recognized character information, determine the first matching degree output by the first recognition model;

[0061] Step S202: Based on the first recognized character information and the second recognized character information, determine the second matching degree output by the second recognition model, wherein the first matching degree and the second matching degree are used to measure the consistency between the first recognition result and the second recognition result;

[0062] Step S203: Determine the preset first parameter and the preset second parameter;

[0063] Step S204: Establish a screening model based on the first confidence level, the second confidence level, the first matching degree, the second matching degree, the preset first parameter, and the preset second parameter.

[0064] An optional step of establishing a screening model based on a first confidence level, a second confidence level, a first matching degree, a second matching degree, a preset first parameter, and a preset second parameter includes: calculating a first sum value 1-conA+ε1 between the preset first parameter, 1, and the opposite of the first confidence level; using a log function, calculating a first function value log(MA+ε2) of the sum of the first matching degree and the preset second parameter; calculating a first product value (1-conA+ε1)log(MA+ε2) between the first sum value and the first function value; and calculating a second sum value 1-conA+ε1 between the preset first parameter, 1, and the opposite of the second confidence level. conB+ε1; Using the log function, calculate the second function value log(MB+ε2) of the sum of the second matching degree and the preset second parameter; calculate the second product value (1-conB+ε1)log(MB+ε2) between the second sum and the second function value; accumulate the negative of the first product value and the negative of the second product value -(1-conA+ε1)log(MA+ε2)-(1-conB+ε1)log(MB+ε2) to obtain the screening degree model Ch=-(1-conA+ε1)log(MA+ε2)-(1-conB+ε1)log(MB+ε2).

[0065] Optionally, the step of determining the first matching degree output by the first recognition model based on the first recognition character information and the second recognition character information includes: matching the first recognition character information and the second recognition character information to obtain the number of identical characters; determining the total number of first characters recognized by the first recognition model based on the first recognition character information; calculating the first ratio between the number of identical characters and the total number of first characters; and representing the first ratio as the first matching degree.

[0066] Optionally, the step of determining the second matching degree output by the second recognition model based on the first and second recognition character information includes: matching the first and second recognition character information to obtain the number of identical characters; determining the total number of second characters recognized by the second recognition model based on the second recognition character information; calculating the second ratio between the number of identical characters and the total number of second characters; and representing the second ratio as the second matching degree.

[0067] In this embodiment of the invention, the matching degree M is calculated as the quotient of the number of identical characters that can be matched in the recognition results of the first recognition model and the second recognition model and the total number of characters in their respective recognition results.

[0068] Therefore, the first matching degree MA is calculated as: n = the number of characters that are consistent between the two recognition models / N = the number of characters in the recognition result of the first recognition model. A(That is, first match the first and second recognition character information to obtain the number of identical characters, then determine the total number of first characters recognized by the first recognition model based on the first recognition character information, and then calculate the first ratio between the number of identical characters and the total number of first characters, and represent the first ratio as the first matching degree).

[0069] The second matching degree (MB) is calculated as follows: (n) the number of characters that are identical in the recognition results of the two recognition models / (N) the number of characters in the recognition result of the second recognition model. B (That is, first match the first and second recognition character information to obtain the number of identical characters, then determine the total number of second characters recognized by the second recognition model based on the second recognition character information, and then calculate the second ratio between the number of identical characters and the total number of second characters, and represent the second ratio as the second matching degree).

[0070] Step S104: Based on the screening degree model, determine the screening degree value of each image slice, and based on the screening degree value, screen the image slice set to obtain the target image slice set.

[0071] Optionally, the step of filtering the image slice set based on the filtering degree value to obtain the target image slice set includes: sorting the filtering degree values ​​to obtain a sorting result; and filtering the image slices indicated by the filtering degree values ​​within a preset range based on the sorting result to obtain the target image slice set, wherein all filtering degree values ​​within the preset range are greater than the filtering degree threshold.

[0072] In this embodiment of the invention, the screening degree value of each image slice can be determined by the screening degree model. Then, the image slice set is filtered according to the screening degree value to obtain the target image slice set to be labeled. Specifically, the screening degree values ​​can be sorted first to obtain the sorting result. Then, according to the sorting result, the image slices indicated by the screening degree value that are within a preset range (such as the top 30%, where all screening degree values ​​within the preset range are greater than the screening degree threshold) are filtered to obtain the target image slice set.

[0073] In this embodiment, an image slice with a high screening score indicates that the current recognition effect of the image slice is not good and it needs to be re-labeled.

[0074] In this embodiment, a screening threshold can be preset, and then the preset range to be screened can be determined based on the screening value of each image slice. This allows us to determine which image slices need to be labeled and which do not, thus improving the flexibility of the screening process.

[0075] Step S105: Label each target image slice in the target image slice set.

[0076] Optionally, the step of annotating each target image slice in the target image slice set includes: obtaining initial character information for annotating each target image slice based on the first recognition character information and the second recognition character information; adjusting the corresponding initial character information according to each target image slice to complete the annotation of the target image slice, wherein each annotated target image slice corresponds to character information.

[0077] In this embodiment of the invention, an initial result for labeling image slices can be obtained based on the recognition character information in the recognition results of the two recognition models (i.e., based on the first recognition character information and the second recognition character information, the initial character information for labeling each target image slice is obtained). Then, according to each target image slice, the corresponding initial character information is adjusted sequentially to complete the labeling of the target image slices and obtain the final labeling result (i.e., each target image slice that has been labeled has corresponding character information).

[0078] In this embodiment, image slices with high screening scores can be selected and labeled simultaneously, which can shorten the labeling time and save time and costs.

[0079] Optionally, after labeling each target image slice in the target image slice set, the method further includes: for image slices with a screening degree value less than or equal to the screening degree threshold, using the character information in the first recognition result or the second recognition result as the character information of the image slice; representing all image slices with character information and all target image slices as a training sample set; and using the training sample set to train the recognition model to be trained until the recognition accuracy of the recognition model to be trained is greater than a preset accuracy threshold.

[0080] In this embodiment of the invention, image slices with low screening scores indicate that the current recognition effect of the image slice is good, and they can be directly used as the final labeling result (i.e., for image slices with screening scores less than or equal to the screening score threshold, the character information in the first or second recognition result is used as the character information of the image slice). Then, all image slices with character information and all target image slices are represented as a training sample set. The training sample set is used to train the recognition model until the recognition accuracy of the model exceeds a preset accuracy threshold (i.e., the recognition accuracy of the model reaches a practical standard).

[0081] The following describes in detail another optional implementation method.

[0082] Figure 3 This is a schematic diagram of an optional image slice identification information annotation process according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following:

[0083] (1) Input the image slice to be labeled, two trained recognition models A and B, and the model to be trained.

[0084] (2) Recognition models A and B are used to identify the image slices to be labeled, and the recognition information RecA and RecB, and the confidence of the recognition information conA and conB are obtained.

[0085] (3) Calculate the matching degree MA and MB based on the identification information RecA and Rec.

[0086] (4) Calculate the screening value of each image slice to be labeled based on the confidence scores conA and conB, and the matching scores MA and MB.

[0087] (5) Determine whether the screening degree value is greater than the preset screening threshold.

[0088] (6) Re-annotate the image slices to be labeled with a screening degree value greater than the preset screening threshold to obtain the labeling results; and directly use the identification information of the image slices to be labeled with a screening degree value less than or equal to the preset screening threshold as the labeling results.

[0089] (7) Use image slices with labeled results as training samples to directly train the model to be trained until the training accuracy reaches the practical standard.

[0090] (8) Output the labeled image slices and the trained recognition model.

[0091] In this embodiment of the invention, a screening degree model is constructed by fusing the confidence scores of different recognition models and the matching degree between models to screen out the image slices that need to be labeled. This effectively reduces the workload of image slice labeling, saves labeling costs, and solves the problem of huge cost of dataset labeling in the field of image text recognition. It can achieve the following beneficial effects: (1) It does not require labeling all image slices one by one, which greatly reduces the workload of labeling; (2) It can screen out image slices with poor recognition effect, poor matching degree, and low confidence, which reduces the workload of labeling, improves work efficiency, and promotes the labeling process towards saving time and effort, improving refinement, improving efficiency, and reducing costs; (3) By sorting, the selected image slices can be labeled first and used for training first, which also significantly improves the model recognition effect; (4) By identifying and screening at the same time, the image slices with high screening degree values ​​can be placed on top and checked and labeled first. For image slices with low screening degree values, their recognition information can be directly used as the real label of the image slice, which can reduce the number of labeled image slices. In addition, for image slices with high screening scores, the recognition results of the two recognition models can be used as two non-real labels for the annotators to review and modify, instead of the annotators manually adding labels, which can reduce the workload of annotation.

[0092] The following is a detailed description with reference to another embodiment.

[0093] Example 2

[0094] The image slicing-based annotation device provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in Embodiment 1 above.

[0095] Figure 4 This is a schematic diagram of an optional image tile-based annotation device according to an embodiment of the present invention, such as... Figure 4 As shown, the annotation device may include: a receiving unit 40, an identification unit 41, an establishment unit 42, a filtering unit 43, and an annotation unit 44, wherein...

[0096] The receiving unit 40 is configured to receive an image tile annotation request, wherein the image tile annotation request includes at least: an image tile set;

[0097] The recognition unit 41 is used to recognize each image slice in the image slice set using a first recognition model to obtain a first recognition result, and to recognize each image slice in the image slice set using a second recognition model to obtain a second recognition result. The first recognition model and the second recognition model are both pre-trained recognition models. The first recognition result includes at least: a first confidence level and first recognition character information. The second recognition result includes at least: a second confidence level and second recognition character information.

[0098] Establishment unit 42 is used to establish a screening degree model based on the first confidence level, the first identified character information, the second confidence level, and the second identified character information;

[0099] The filtering unit 43 is used to determine the filtering degree value of each image slice based on the filtering degree model, and to filter the image slice set based on the filtering degree value to obtain the target image slice set.

[0100] The annotation unit 44 is used to annotate each target image slice in the target image slice set.

[0101] The above-mentioned annotation device can receive image slice annotation requests through receiving unit 40, identify each image slice in the image slice set using a first recognition model through recognition unit 41 to obtain a first recognition result, and identify each image slice in the image slice set using a second recognition model to obtain a second recognition result, establish a screening degree model through establishing unit 42 based on a first confidence level, a first recognition character information, a second confidence level, and a second recognition character information, determine the screening degree value of each image slice based on the screening degree model through screening unit 43, and screen the image slice set based on the screening degree value to obtain a target image slice set, and annotate each target image slice in the target image slice set through annotation unit 44. In this embodiment of the invention, a screening degree model can be established based on the first confidence level and first recognition character information identified by the first recognition model for the image slice, and the second confidence level and second recognition character information identified by the second recognition model for the image slice. The screening degree model determines the screening degree value of each image slice, and then the target image slices that need to be labeled are selected based on the screening degree value. This can reduce the number of image slices that need to be labeled, effectively improve the labeling efficiency, reduce the cost, and thus solve the technical problem of low efficiency and high cost in related technologies that rely solely on manual labeling of a large number of image slices.

[0102] Optionally, the establishment unit includes: a first determining module, used to determine a first matching degree output by a first recognition model based on first and second recognition character information; a second determining module, used to determine a second matching degree output by a second recognition model based on the first and second recognition character information, wherein the first and second matching degrees are used to measure the consistency between the first and second recognition results; a third determining module, used to determine preset first and preset second parameters; and a first establishing module, used to establish a screening degree model based on the first confidence level, the second confidence level, the first matching degree, the second matching degree, the preset first parameter, and the preset second parameter.

[0103] Optionally, the first determining module includes: a first matching submodule, used to match the first identified character information and the second identified character information to obtain the number of identical characters; a first determining submodule, used to determine the total number of first characters identified by the first identification model based on the first identified character information; a first calculation submodule, used to calculate a first ratio between the number of identical characters and the total number of first characters; and a first characterization submodule, used to characterize the first ratio as a first matching degree.

[0104] Optionally, the second determining module includes: a second matching submodule, used to match the first identified character information and the second identified character information to obtain the number of identical characters; a second determining submodule, used to determine the total number of second characters identified by the second identification model based on the second identified character information; a second calculation submodule, used to calculate a second ratio between the number of identical characters and the total number of second characters; and a second characterization submodule, used to characterize the second ratio as a second matching degree.

[0105] Optionally, the filtering unit includes: a first sorting module for sorting the filtering degree values ​​to obtain a sorting result; and a first filtering module for filtering image slices indicated by filtering degree values ​​within a preset range based on the sorting result to obtain a target image slice set, wherein all filtering degree values ​​within the preset range are greater than the filtering degree threshold.

[0106] Optionally, the annotation unit includes: a first output module, used to obtain initial character information for annotating each target image slice based on the first and second recognition character information; and a first adjustment module, used to adjust the corresponding initial character information according to each target image slice to complete the annotation of the target image slice, wherein each annotated target image slice corresponds to character information.

[0107] Optionally, the annotation device further includes: a first assignment module, used to, after annotating each target image slice in the target image slice set, use the character information in the first recognition result or the second recognition result as the character information of the image slice for image slices with a screening degree value less than or equal to the screening degree threshold; a first representation module, used to represent all image slices with character information and all target image slices as a training sample set; and a first training module, used to train the recognition model to be trained using the training sample set until the recognition accuracy of the recognition model to be trained is greater than a preset accuracy threshold.

[0108] The labeling device described above may also include a processor and a memory. The receiving unit 40, the identification unit 41, the establishment unit 42, the filtering unit 43, the labeling unit 44, etc., are all stored in the memory as program units, and the processor executes the program units stored in the memory to realize the corresponding functions.

[0109] The processor described above contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and the kernel parameters are adjusted to annotate each target image slice in the target image slice set.

[0110] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0111] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: receiving an image slice annotation request, using a first recognition model to identify each image slice in the image slice set to obtain a first recognition result, and using a second recognition model to identify each image slice in the image slice set to obtain a second recognition result, establishing a screening degree model based on a first confidence level, first recognition character information, a second confidence level, and second recognition character information, determining a screening degree value for each image slice based on the screening degree model, and screening the image slice set based on the screening degree value to obtain a target image slice set, and annotating each target image slice in the target image slice set.

[0112] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the above-described image tile-based annotation method.

[0113] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the above-described image tile-based annotation method.

[0114] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) for an image tile-based annotation method according to an embodiment of the present invention. Figure 5 As shown, the electronic device may include one or more processors 502 (shown as 502a, 502b, ..., 502n in the figure) 502 (processor 502 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 504 for storing data. In addition, it may include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 5 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 5 The more or fewer components shown, or having the sameFigure 5 The different configurations shown.

[0115] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0116] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0120] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0121] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A labeling method based on image slicing, characterized in that, include: Receive an image slice annotation request, wherein the image slice annotation request includes at least: a set of image slices; Each image slice in the image slice set is identified using a first recognition model to obtain a first recognition result, and each image slice in the image slice set is identified using a second recognition model to obtain a second recognition result. Both the first recognition model and the second recognition model are pre-trained recognition models. The first recognition result includes at least: a first confidence level and first recognition character information, and the second recognition result includes at least: a second confidence level and second recognition character information. A screening model is established based on the first confidence level, the first identified character information, the second confidence level, and the second identified character information; Based on the filtering degree model, the filtering degree value of each image slice is determined, and based on the filtering degree value, the image slice set is filtered to obtain the target image slice set; Each target image slice in the target image slice set is labeled; The step of establishing a screening model based on the first confidence level, the first identified character information, the second confidence level, and the second identified character information includes: Based on the first recognized character information and the second recognized character information, the first matching degree output by the first recognition model is determined; Based on the first recognized character information and the second recognized character information, a second matching degree is determined from the output of the second recognition model, wherein the first matching degree and the second matching degree are used to measure the consistency between the first recognition result and the second recognition result; Determine the first preset parameter and the second preset parameter; The screening model is established based on the first confidence level, the second confidence level, the first matching degree, the second matching degree, the preset first parameter, and the preset second parameter.

2. The annotation method according to claim 1, characterized in that, The step of determining the first matching degree output by the first recognition model based on the first recognized character information and the second recognized character information includes: Match the first identified character information and the second identified character information to obtain the same number of characters; Based on the first recognized character information, determine the total number of first characters recognized by the first recognition model; Calculate the first ratio between the number of identical characters and the total number of the first characters; The first ratio is represented as the first matching degree.

3. The annotation method according to claim 1, characterized in that, The step of determining the second matching degree output by the second recognition model based on the first and second recognized character information includes: Match the first identified character information and the second identified character information to obtain the same number of characters; Based on the second recognized character information, determine the total number of second characters recognized by the second recognition model; Calculate the second ratio between the number of identical characters and the total number of the second characters; The second ratio is represented as the second matching degree.

4. The annotation method according to claim 1, characterized in that, The step of filtering the image slice set based on the filtering degree value to obtain the target image slice set includes: The screening scores are sorted to obtain the sorting results; Based on the sorting results, the image slices indicated by the filtering degree values ​​within a preset range are filtered to obtain the target image slice set, wherein all the filtering degree values ​​within the preset range are greater than the filtering degree threshold.

5. The annotation method according to claim 1, characterized in that, The step of annotating each target image slice in the target image slice set includes: Based on the first and second recognized character information, initial character information is obtained to annotate each of the target image slices; Based on each target image slice, the corresponding initial character information is adjusted to complete the annotation of the target image slice, wherein each annotated target image slice corresponds to character information.

6. The annotation method according to claim 5, characterized in that, After labeling each target image slice in the target image slice set, the method further includes: For image slices whose filtering degree value is less than or equal to the filtering degree threshold, the character information in the first recognition result or the second recognition result is used as the character information of the image slice; All the image slices containing character information and all the target image slices are represented as a training sample set; The training sample set is used to train the recognition model to be trained until the recognition accuracy of the recognition model to be trained is greater than a preset accuracy threshold.

7. An image slicing-based annotation device, characterized in that, include: A receiving unit is configured to receive an image slice annotation request, wherein the image slice annotation request includes at least: an image slice set; The recognition unit is configured to recognize each image slice in the image slice set using a first recognition model to obtain a first recognition result, and to recognize each image slice in the image slice set using a second recognition model to obtain a second recognition result, wherein the first recognition model and the second recognition model are both pre-trained recognition models, the first recognition result includes at least: a first confidence level and first recognition character information, and the second recognition result includes at least: a second confidence level and second recognition character information; The establishment unit is used to establish a screening model based on the first confidence level, the first identified character information, the second confidence level, and the second identified character information; A filtering unit is used to determine the filtering degree value of each image slice based on the filtering degree model, and to filter the image slice set based on the filtering degree value to obtain a target image slice set. Annotation unit, used to annotate each target image slice in the target image slice set; The establishment unit includes: a first determining module, configured to determine a first matching degree output by the first recognition model based on the first recognition character information and the second recognition character information; a second determining module, configured to determine a second matching degree output by the second recognition model based on the first recognition character information and the second recognition character information, wherein the first matching degree and the second matching degree are used to measure the consistency between the first recognition result and the second recognition result; a third determining module, configured to determine a preset first parameter and a preset second parameter; and a first establishing module, configured to establish the screening degree model based on the first confidence level, the second confidence level, the first matching degree, the second matching degree, the preset first parameter, and the preset second parameter.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the image tile-based annotation method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the image tile-based annotation method according to any one of claims 1 to 6.

Images